[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:02:51 dist-test-slave-j8c1 krb5kdc[809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:02:51 dist-test-slave-j8c1 krb5kdc[809](info): set up 2 sockets
Sep 26 04:02:51 dist-test-slave-j8c1 krb5kdc[809](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 04:02:52.915607 802 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:02:53.042399 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37727
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:53.124908 822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:53.125190 822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:53.125231 822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:53.126509 822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:02:53.126636 822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:53.126684 822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:53.126714 822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:02:53.126740 822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:02:53.128450 822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:53.128779 822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:53.129050 822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:02:53.134472 822 server_base.cc:1034] running on GCE node
W20260926 04:02:53.134439 827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.134483 828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.134434 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:53.134893 822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:53.135251 822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:53.136421 822 hybrid_clock.cc:648] HybridClock initialized: now 1790395373136404 us; error 47 us; skew 500 ppm
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:53.139274 822 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:02:53.139840 822 webserver.cc:460] Webserver started at http://127.0.200.190:38099/ using document root <none> and password file <none>
I20260926 04:02:53.140136 822 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:53.140208 822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:53.140354 822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:53.141477 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "2828ba4b4aa3497bb71d02662ff0ba83"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "d5ee13ff23dcd4bede3a7de274b32927"
server_key_iv: "c8a529bca8bdaaeebd05666262205b2d"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.141880 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "2828ba4b4aa3497bb71d02662ff0ba83"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "d5ee13ff23dcd4bede3a7de274b32927"
server_key_iv: "c8a529bca8bdaaeebd05666262205b2d"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.143775 822 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260926 04:02:53.144721 837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.144997 822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:02:53.145113 822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "2828ba4b4aa3497bb71d02662ff0ba83"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "d5ee13ff23dcd4bede3a7de274b32927"
server_key_iv: "c8a529bca8bdaaeebd05666262205b2d"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.145215 822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:53.156126 822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:53.158650 822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:53.158829 822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:53.163400 822 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:37727
I20260926 04:02:53.163440 889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:37727 every 8 connection(s)
I20260926 04:02:53.163828 822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:02:53.164753 890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:53.167294 890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83: Bootstrap starting.
I20260926 04:02:53.167907 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 822
I20260926 04:02:53.168028 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:02:53.168052 890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:53.168381 890 log.cc:824] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:53.168790 802 external_mini_cluster.cc:1428] Setting key ffc439d509f6fe94f41057c85e99030d
I20260926 04:02:53.169217 890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83: No bootstrap required, opened a new log
I20260926 04:02:53.171316 890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } }
I20260926 04:02:53.171525 890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:53.171574 890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2828ba4b4aa3497bb71d02662ff0ba83, State: Initialized, Role: FOLLOWER
I20260926 04:02:53.171738 890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } }
I20260926 04:02:53.171829 890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:02:53.171872 890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:02:53.171922 890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:53.173219 890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } }
I20260926 04:02:53.173353 890 leader_election.cc:302] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2828ba4b4aa3497bb71d02662ff0ba83; no voters:
I20260926 04:02:53.173552 890 leader_election.cc:288] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:02:53.173578 895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:53.173703 895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [term 1 LEADER]: Becoming Leader. State: Replica: 2828ba4b4aa3497bb71d02662ff0ba83, State: Running, Role: LEADER
I20260926 04:02:53.173787 895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } }
I20260926 04:02:53.173840 890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:02:53.174160 895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2828ba4b4aa3497bb71d02662ff0ba83. Latest consensus state: current_term: 1 leader_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } } }
I20260926 04:02:53.174168 897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2828ba4b4aa3497bb71d02662ff0ba83" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37727 } } }
I20260926 04:02:53.174245 895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [sys.catalog]: This master's current role is: LEADER
I20260926 04:02:53.174269 897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83 [sys.catalog]: This master's current role is: LEADER
I20260926 04:02:53.174603 901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:02:53.175102 901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:02:53.176837 901 catalog_manager.cc:1347] Generated new cluster ID: 452586715ec74352beb2e17d7dec6f8b
I20260926 04:02:53.176896 901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:53.188556 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.169782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.0.200.190:37727)
0926 04:02:53.169998 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.170008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.170224 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:02:53.172307 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.172312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.172357 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.172496 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.174411 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.175851 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.177400 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.178006 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.178115 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.182639 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.182643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.182644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.182658 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.185435 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.185996 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.186030 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.186031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.186219 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.187151 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.187152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.187152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.187498 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.187666 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.187971 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.188120 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260926 04:02:53.193935 901 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:02:53.195029 901 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:02:53.200415 901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2828ba4b4aa3497bb71d02662ff0ba83: Generated new TSK 0
I20260926 04:02:53.200711 901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:02:53.233719 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:53.321511 916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:53.321751 916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:53.321777 916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:53.323118 916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:53.323167 916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:53.323196 916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:02:53.324783 916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:53.325060 916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:53.325320 916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:53.330765 922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:53.330710 916 server_base.cc:1034] running on GCE node
W20260926 04:02:53.330710 924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.330736 921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:53.331142 916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:53.331359 916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:53.332515 916 hybrid_clock.cc:648] HybridClock initialized: now 1790395373332485 us; error 43 us; skew 500 ppm
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:53.335421 916 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:02:53.335911 916 webserver.cc:460] Webserver started at http://127.0.200.129:44651/ using document root <none> and password file <none>
I20260926 04:02:53.336120 916 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:53.336164 916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:53.336256 916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:53.337297 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "e0ae6700c1014fb2b10ca12669bc9b56"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "8a2437a16f7fb96d4807a9aa8ebdcbe8"
server_key_iv: "d5e78f397f3f64774c8074dad7e0fc1b"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.337605 916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "e0ae6700c1014fb2b10ca12669bc9b56"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "8a2437a16f7fb96d4807a9aa8ebdcbe8"
server_key_iv: "d5e78f397f3f64774c8074dad7e0fc1b"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.339036 916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:02:53.339828 931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.340045 916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:02:53.340164 916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "e0ae6700c1014fb2b10ca12669bc9b56"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "8a2437a16f7fb96d4807a9aa8ebdcbe8"
server_key_iv: "d5e78f397f3f64774c8074dad7e0fc1b"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.340235 916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:53.349920 916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:53.352324 916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:53.352444 916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:53.353026 916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:53.353284 916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:53.353315 916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.353338 916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:53.353349 916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.359695 916 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:40121
I20260926 04:02:53.359784 1044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:40121 every 8 connection(s)
I20260926 04:02:53.360113 916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:53.369019 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 916
I20260926 04:02:53.369115 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:02:53.369211 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.360917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44599 (local address 127.0.200.190:37727)
0926 04:02:53.360979 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.360981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.361154 (+ 173us) server_negotiation.cc:407] Connection header received
0926 04:02:53.361631 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.361632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.361644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.361705 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.362683 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.363192 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.364191 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.364375 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.364438 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.366478 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.366479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.366479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.366486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.367746 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.368143 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.368144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.368144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.368198 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.368456 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.368457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.368458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.368602 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.368668 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.369004 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.369076 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 04:02:53.369277 802 external_mini_cluster.cc:1428] Setting key a00e1d8b45559347622d8380a497e1c2
I20260926 04:02:53.369479 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.360863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37727 (local address 127.0.200.129:44599)
0926 04:02:53.361027 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:53.361069 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:53.361549 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:53.361757 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:53.361760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:53.362338 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:53.362536 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.362541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.363281 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.363282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.364111 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.364113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.364456 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.364456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.364552 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.365035 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:53.365043 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:53.366385 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:53.367815 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.367821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.367823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.368073 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.368256 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.368257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.368257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.368350 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.368674 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:53.368678 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:53.368800 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 04:02:53.368986 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 04:02:53.369172 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260926 04:02:53.370016 1045 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37727
I20260926 04:02:53.370108 1045 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:53.370301 1045 heartbeater.cc:511] Master 127.0.200.190:37727 requested a full tablet report, sending...
I20260926 04:02:53.370924 854 ts_manager.cc:194] Registered new tserver with Master: e0ae6700c1014fb2b10ca12669bc9b56 (127.0.200.129:40121)
I20260926 04:02:53.371753 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:44599
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:02:53.397308 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:53.481117 1050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:53.481370 1050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:53.481410 1050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:53.482630 1050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:53.482698 1050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:53.482745 1050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:02:53.484323 1050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:53.484622 1050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:53.484866 1050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:53.490095 1055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.490075 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.490173 1058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:53.490473 1050 server_base.cc:1034] running on GCE node
I20260926 04:02:53.490751 1050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:53.491024 1050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:53.492167 1050 hybrid_clock.cc:648] HybridClock initialized: now 1790395373492156 us; error 28 us; skew 500 ppm
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:53.494726 1050 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:02:53.495137 1050 webserver.cc:460] Webserver started at http://127.0.200.130:40261/ using document root <none> and password file <none>
I20260926 04:02:53.495342 1050 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:53.495384 1050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:53.495476 1050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:53.496505 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "4dd7ba950a424755b86c78dae3a53918"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "b35d89f8aeb39733c91c5208863191be"
server_key_iv: "f76ee9b43ca8834e139aa3e31fe98595"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.496819 1050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "4dd7ba950a424755b86c78dae3a53918"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "b35d89f8aeb39733c91c5208863191be"
server_key_iv: "f76ee9b43ca8834e139aa3e31fe98595"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.498191 1050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:02:53.499202 1065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.499418 1050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:02:53.499505 1050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "4dd7ba950a424755b86c78dae3a53918"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "b35d89f8aeb39733c91c5208863191be"
server_key_iv: "f76ee9b43ca8834e139aa3e31fe98595"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.499589 1050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:53.506966 1050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:53.509248 1050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:53.509371 1050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:53.509547 1050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:53.509881 1050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:53.509918 1050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.509943 1050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:53.509954 1050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.516086 1050 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:35061
I20260926 04:02:53.516176 1178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:35061 every 8 connection(s)
I20260926 04:02:53.516495 1050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:02:53.522177 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1050
I20260926 04:02:53.522280 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:02:53.522491 802 external_mini_cluster.cc:1428] Setting key 9977a3d28499bd19e3367822ac1bbb94
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:53.526922 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.517262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:36481 (local address 127.0.200.190:37727)
0926 04:02:53.517321 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.517323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.517536 (+ 213us) server_negotiation.cc:407] Connection header received
0926 04:02:53.518011 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.518013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.518024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.518087 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.519167 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.519674 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.520708 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.520902 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.520963 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.523702 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.523703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.523704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.523710 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.525166 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.525716 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.525717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.525718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.525781 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.526104 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.526105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.526106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.526226 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.526302 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.526706 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.526807 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:02:53.527148 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.517223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37727 (local address 127.0.200.130:36481)
0926 04:02:53.517409 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:53.517450 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:53.517926 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:53.518150 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:53.518153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:53.518857 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:53.519058 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.519064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.519768 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.519769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.520614 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.520616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.521007 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.521008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.521121 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.521688 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:53.521696 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:53.523602 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:53.525248 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.525255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.525258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.525614 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.525855 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.525856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.525857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.526010 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.526301 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:53.526307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:53.526495 (+ 188us) client_negotiation.cc:769] Sending connection context
0926 04:02:53.526655 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 04:02:53.526858 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"spinlock_wait_cycles":8448,"thread_start_us":110,"threads_started":1}
I20260926 04:02:53.527676 1179 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37727
I20260926 04:02:53.527804 1179 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:53.528026 1179 heartbeater.cc:511] Master 127.0.200.190:37727 requested a full tablet report, sending...
I20260926 04:02:53.528443 854 ts_manager.cc:194] Registered new tserver with Master: 4dd7ba950a424755b86c78dae3a53918 (127.0.200.130:35061)
I20260926 04:02:53.528820 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:36481
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:02:53.550271 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:53.634502 1184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:53.634776 1184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:53.634821 1184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:53.636075 1184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:53.636145 1184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:53.636193 1184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:02:53.637745 1184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:53.638041 1184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:53.638298 1184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:53.644037 1190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.643997 1189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:53.644019 1192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:53.644354 1184 server_base.cc:1034] running on GCE node
I20260926 04:02:53.644629 1184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:53.644884 1184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:53.646062 1184 hybrid_clock.cc:648] HybridClock initialized: now 1790395373646055 us; error 37 us; skew 500 ppm
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:53.649003 1184 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:02:53.649505 1184 webserver.cc:460] Webserver started at http://127.0.200.131:37405/ using document root <none> and password file <none>
I20260926 04:02:53.649755 1184 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:53.649801 1184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:53.649935 1184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:53.651090 1184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "664d1f455d2344c49e0ce1659a01e27c"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "701aeb689c8247c1dd540f2580ea2605"
server_key_iv: "b858906fbd294c48a67aed2283ca9811"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.651463 1184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "664d1f455d2344c49e0ce1659a01e27c"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "701aeb689c8247c1dd540f2580ea2605"
server_key_iv: "b858906fbd294c48a67aed2283ca9811"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.652871 1184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 04:02:53.653626 1199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.653797 1184 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:02:53.653874 1184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "664d1f455d2344c49e0ce1659a01e27c"
format_stamp: "Formatted at 2026-09-26 04:02:53 on dist-test-slave-j8c1"
server_key: "701aeb689c8247c1dd540f2580ea2605"
server_key_iv: "b858906fbd294c48a67aed2283ca9811"
server_key_version: "encryptionkey@0"
I20260926 04:02:53.653960 1184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:53.670382 1184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:53.672947 1184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:53.673116 1184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:53.673401 1184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:53.673874 1184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:53.673969 1184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.674015 1184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:53.674089 1184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:53.682410 1184 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:38567
I20260926 04:02:53.682446 1312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:38567 every 8 connection(s)
I20260926 04:02:53.682819 1184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:02:53.686106 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1184
I20260926 04:02:53.686208 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:02:53.686399 802 external_mini_cluster.cc:1428] Setting key 5a30c142b6a86debf77e250faac00c2f
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:53.693251 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.683662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:44433 (local address 127.0.200.190:37727)
0926 04:02:53.683707 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.683710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.683959 (+ 249us) server_negotiation.cc:407] Connection header received
0926 04:02:53.684449 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.684451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.684460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.684536 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.685557 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.686182 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.687694 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.687882 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.687940 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.690341 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.690342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.690343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.690351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.691807 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.692255 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.692256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.692256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.692316 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.692582 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.692583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.692583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.692707 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.692781 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.693073 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.693152 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:02:53.693464 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.683610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37727 (local address 127.0.200.131:44433)
0926 04:02:53.683771 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:53.683827 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:53.684363 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:53.684611 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:53.684615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:53.685267 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:53.685470 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.685475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.686307 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.686308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.687612 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:53.687616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.687966 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.687967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:53.688072 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.688695 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:53.688704 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:53.690227 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:53.691911 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.691920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.691922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.692165 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.692381 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:53.692382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:53.692382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:53.692509 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:53.692772 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:53.692778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:53.692931 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 04:02:53.693038 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 04:02:53.693213 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260926 04:02:53.693993 1313 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37727
I20260926 04:02:53.694103 1313 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:53.694273 1313 heartbeater.cc:511] Master 127.0.200.190:37727 requested a full tablet report, sending...
I20260926 04:02:53.694698 854 ts_manager.cc:194] Registered new tserver with Master: 664d1f455d2344c49e0ce1659a01e27c (127.0.200.131:38567)
I20260926 04:02:53.695089 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:44433
I20260926 04:02:53.698499 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:53.722604 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.714429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46898 (local address 127.0.200.129:40121)
0926 04:02:53.714690 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.714694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.714706 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:02:53.714733 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.714735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.714749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.714854 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.715421 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.716167 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.716891 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.717336 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.717394 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.719767 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.719774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.719774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.719780 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.721417 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.721768 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.721772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.721772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.721826 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.721995 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.721996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.721996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.722134 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.722255 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.722389 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.722452 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":93,"threads_started":1}
W20260926 04:02:53.723073 959 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46898
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:53.747264 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:53.739539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.0.200.129:40121)
0926 04:02:53.739628 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:53.739629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:53.739659 (+ 30us) server_negotiation.cc:407] Connection header received
0926 04:02:53.739731 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:53.739732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:53.739745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:53.739806 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:53.740403 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.740979 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.741744 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:53.742002 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:53.742067 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:53.744537 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:53.744538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:53.744539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:53.744545 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:53.745966 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.746322 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.746323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.746323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.746394 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:53.746728 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:53.746728 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:53.746729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:53.746848 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:53.746990 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:53.747093 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 04:02:53.747166 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 04:02:53.747905 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 916
I20260926 04:02:53.753500 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1050
I20260926 04:02:53.759193 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1184
I20260926 04:02:53.763975 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 822
2026-09-26T04:02:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1939 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[1327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[1327](info): set up 2 sockets
Sep 26 04:02:53 dist-test-slave-j8c1 krb5kdc[1327](info): commencing operation
krb5kdc: starting...
W20260926 04:02:54.826406 802 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:02:54 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:02:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:02:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:02:54.953886 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:42425
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:42425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:55.037511 1340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:55.037798 1340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:55.037842 1340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:55.039110 1340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:02:55.039189 1340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:55.039215 1340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:55.039245 1340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:02:55.039274 1340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:02:55.040823 1340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:42425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:42425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:55.041115 1340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:55.041363 1340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:55.046365 1346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.046373 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.046445 1340 server_base.cc:1034] running on GCE node
W20260926 04:02:55.046433 1348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.046947 1340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:55.047232 1340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:55.048395 1340 hybrid_clock.cc:648] HybridClock initialized: now 1790395375048371 us; error 39 us; skew 500 ppm
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:55.051056 1340 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:02:55.051568 1340 webserver.cc:460] Webserver started at http://127.0.200.190:41443/ using document root <none> and password file <none>
I20260926 04:02:55.051818 1340 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:55.051882 1340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:55.052035 1340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:55.053188 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "db4ff2d3633434be081459983a25beda"
server_key_iv: "31c1b54aa2508f772d324d0ef6a947ae"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.053567 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "db4ff2d3633434be081459983a25beda"
server_key_iv: "31c1b54aa2508f772d324d0ef6a947ae"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.055039 1340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:02:55.056129 1355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.056416 1340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:02:55.056571 1340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "db4ff2d3633434be081459983a25beda"
server_key_iv: "31c1b54aa2508f772d324d0ef6a947ae"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.056658 1340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:55.067756 1340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:55.070055 1340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:55.070246 1340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:55.074712 1340 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:42425
I20260926 04:02:55.074751 1407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:42425 every 8 connection(s)
I20260926 04:02:55.075210 1340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:02:55.076149 1408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:55.078506 1408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: Bootstrap starting.
I20260926 04:02:55.079142 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1340
I20260926 04:02:55.079182 1408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:55.079252 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:02:55.079407 802 external_mini_cluster.cc:1428] Setting key f165d8f9491e1e94223e73b2100f94f0
I20260926 04:02:55.079509 1408 log.cc:824] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:55.080334 1408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: No bootstrap required, opened a new log
I20260926 04:02:55.082677 1408 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } }
I20260926 04:02:55.082849 1408 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:55.082897 1408 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2c733380b04f8fa20d1d8ae9c25a9b, State: Initialized, Role: FOLLOWER
I20260926 04:02:55.083065 1408 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } }
I20260926 04:02:55.083163 1408 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:02:55.083206 1408 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:02:55.083256 1408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:55.083861 1408 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } }
I20260926 04:02:55.083992 1408 leader_election.cc:302] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab2c733380b04f8fa20d1d8ae9c25a9b; no voters:
I20260926 04:02:55.084233 1408 leader_election.cc:288] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:02:55.084263 1413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:55.084443 1413 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [term 1 LEADER]: Becoming Leader. State: Replica: ab2c733380b04f8fa20d1d8ae9c25a9b, State: Running, Role: LEADER
I20260926 04:02:55.084514 1408 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:02:55.085000 1413 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } }
I20260926 04:02:55.085702 1415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } } }
I20260926 04:02:55.085805 1415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [sys.catalog]: This master's current role is: LEADER
I20260926 04:02:55.086339 1414 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab2c733380b04f8fa20d1d8ae9c25a9b. Latest consensus state: current_term: 1 leader_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2c733380b04f8fa20d1d8ae9c25a9b" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42425 } } }
I20260926 04:02:55.086472 1414 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b [sys.catalog]: This master's current role is: LEADER
W20260926 04:02:55.086539 1427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:02:55.086637 1427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:02:55.086700 1428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:55.086867 1428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:02:55.088510 1428 catalog_manager.cc:1347] Generated new cluster ID: 8ee3a04ba0844dd982d3a53e7a2e2ae4
I20260926 04:02:55.088567 1428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:02:55.093413 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.080004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.0.200.190:42425)
0926 04:02:55.080247 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.080258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.080293 (+ 35us) server_negotiation.cc:407] Connection header received
0926 04:02:55.080831 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.080838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.080890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.081058 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.081721 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.082701 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.083642 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.084173 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.084295 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.087397 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.087401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.087402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.087419 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.091603 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.092047 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.092052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.092052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.092181 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.092402 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.092403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.092403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.092756 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.092938 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.093026 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.093187 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260926 04:02:55.100876 1428 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:02:55.101665 1428 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:02:55.112622 1428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab2c733380b04f8fa20d1d8ae9c25a9b: Generated new TSK 0
I20260926 04:02:55.112860 1428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:02:55.149696 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:55.261150 1434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:55.261373 1434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:55.261402 1434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:55.262661 1434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:55.262710 1434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:55.262737 1434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:02:55.264189 1434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:55.264396 1434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:55.264617 1434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:55.270347 1439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.270330 1440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.270501 1442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.270610 1434 server_base.cc:1034] running on GCE node
I20260926 04:02:55.270844 1434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:55.271126 1434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:55.272321 1434 hybrid_clock.cc:648] HybridClock initialized: now 1790395375272315 us; error 39 us; skew 500 ppm
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:55.275194 1434 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:02:55.275681 1434 webserver.cc:460] Webserver started at http://127.0.200.129:33669/ using document root <none> and password file <none>
I20260926 04:02:55.275965 1434 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:55.276031 1434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:55.276163 1434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:55.277246 1434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "50277a38174d41558f6ef031aa5eda1c"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "0355c6772a942408ce6d63e7c621ecdb"
server_key_iv: "ef782625744886048029262b76bd991b"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.277606 1434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "50277a38174d41558f6ef031aa5eda1c"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "0355c6772a942408ce6d63e7c621ecdb"
server_key_iv: "ef782625744886048029262b76bd991b"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.279022 1434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:02:55.279788 1449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.279992 1434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:02:55.280094 1434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "50277a38174d41558f6ef031aa5eda1c"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "0355c6772a942408ce6d63e7c621ecdb"
server_key_iv: "ef782625744886048029262b76bd991b"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.280164 1434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:55.299582 1434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:55.302001 1434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:55.302230 1434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:55.302488 1434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:55.302860 1434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:55.302914 1434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.302958 1434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:55.302983 1434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.308869 1434 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:35599
I20260926 04:02:55.308917 1562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:35599 every 8 connection(s)
I20260926 04:02:55.309247 1434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:02:55.310437 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1434
I20260926 04:02:55.310534 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:02:55.310724 802 external_mini_cluster.cc:1428] Setting key 297fec5d00be0e22e44749cdec0bc6f1
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:02:55.319237 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.309937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:51217 (local address 127.0.200.190:42425)
0926 04:02:55.310015 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.310017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.310232 (+ 215us) server_negotiation.cc:407] Connection header received
0926 04:02:55.310802 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.310804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.310816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.310874 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.312048 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.312566 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.313609 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.313843 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.313929 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.316276 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.316277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.316278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.316286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.317762 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.318210 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.318211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.318212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.318275 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.318542 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.318566 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.318567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.318691 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.318761 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.319038 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.319141 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:02:55.319465 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.309911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42425 (local address 127.0.200.129:51217)
0926 04:02:55.310098 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.310141 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.310696 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.311065 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.311069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.311734 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:55.311944 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.311949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.312682 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.312683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.313516 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.313518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.313939 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.313940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.314066 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.314795 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:55.314804 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:55.316170 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:55.317874 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.317881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.317884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.318129 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.318339 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.318340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.318340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.318476 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.318763 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:55.318769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:55.318912 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.319019 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.319182 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260926 04:02:55.319984 1563 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42425
I20260926 04:02:55.320151 1563 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:55.320353 1563 heartbeater.cc:511] Master 127.0.200.190:42425 requested a full tablet report, sending...
I20260926 04:02:55.320924 1372 ts_manager.cc:194] Registered new tserver with Master: 50277a38174d41558f6ef031aa5eda1c (127.0.200.129:35599)
I20260926 04:02:55.321574 1372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:51217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:02:55.338702 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:55.421767 1568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:55.421994 1568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:55.422021 1568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:55.423354 1568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:55.423410 1568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:55.423440 1568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:02:55.424991 1568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:55.425258 1568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:55.425494 1568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:55.430846 1574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.430881 1576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.431074 1568 server_base.cc:1034] running on GCE node
W20260926 04:02:55.430846 1573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.431344 1568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:55.431602 1568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:55.432794 1568 hybrid_clock.cc:648] HybridClock initialized: now 1790395375432785 us; error 38 us; skew 500 ppm
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:55.435416 1568 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:02:55.435847 1568 webserver.cc:460] Webserver started at http://127.0.200.130:42539/ using document root <none> and password file <none>
I20260926 04:02:55.436038 1568 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:55.436080 1568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:55.436174 1568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:55.437229 1568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "867bb6c8fa2f49baa88721c434e039d7"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "aacb636770790c22842c0beec6c44fe9"
server_key_iv: "38a14d2f30cbbb31c637cd11be32c3e0"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.437542 1568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "867bb6c8fa2f49baa88721c434e039d7"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "aacb636770790c22842c0beec6c44fe9"
server_key_iv: "38a14d2f30cbbb31c637cd11be32c3e0"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.438896 1568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:02:55.439599 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.439783 1568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:02:55.439873 1568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "867bb6c8fa2f49baa88721c434e039d7"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "aacb636770790c22842c0beec6c44fe9"
server_key_iv: "38a14d2f30cbbb31c637cd11be32c3e0"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.439931 1568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:55.451259 1568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:55.453667 1568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:55.453845 1568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:55.454063 1568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:55.454370 1568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:55.454404 1568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.454445 1568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:55.454471 1568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.460537 1568 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:40719
I20260926 04:02:55.460611 1696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:40719 every 8 connection(s)
I20260926 04:02:55.460961 1568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:02:55.464044 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1568
I20260926 04:02:55.464140 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:02:55.464298 802 external_mini_cluster.cc:1428] Setting key 80e1494d5a532608ae0621c4ecee65c3
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:02:55.472159 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.461824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60389 (local address 127.0.200.190:42425)
0926 04:02:55.461904 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.461906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.462162 (+ 256us) server_negotiation.cc:407] Connection header received
0926 04:02:55.462736 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.462737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.462748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.462810 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.463792 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.464619 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.466043 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.466345 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.466432 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.469173 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.469175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.469175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.469189 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.470691 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.471162 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.471163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.471163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.471227 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.471478 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.471480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.471480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.471615 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.471690 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.471971 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.472055 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 04:02:55.472384 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.461806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42425 (local address 127.0.200.130:60389)
0926 04:02:55.461983 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.462027 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.462544 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.462872 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.462875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.463492 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:55.463699 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.463703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.464697 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.464698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.465927 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.465930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.466439 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.466441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.466635 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.467213 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:55.467223 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:55.468989 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:55.470792 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.470800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.470803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.471052 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.471292 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.471292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.471293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.471413 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.471680 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:55.471686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:55.471838 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.471948 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.472131 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260926 04:02:55.472944 1697 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42425
I20260926 04:02:55.473053 1697 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:55.473284 1697 heartbeater.cc:511] Master 127.0.200.190:42425 requested a full tablet report, sending...
I20260926 04:02:55.473752 1372 ts_manager.cc:194] Registered new tserver with Master: 867bb6c8fa2f49baa88721c434e039d7 (127.0.200.130:40719)
I20260926 04:02:55.474200 1372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:60389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:02:55.491832 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:32985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:55.577215 1702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:55.577445 1702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:55.577471 1702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:55.578747 1702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:55.578791 1702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:55.578819 1702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:02:55.580408 1702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:32985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:55.580682 1702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:55.580941 1702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:55.586305 1707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.586261 1708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:55.586285 1710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:55.586647 1702 server_base.cc:1034] running on GCE node
I20260926 04:02:55.586906 1702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:55.587164 1702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:55.588347 1702 hybrid_clock.cc:648] HybridClock initialized: now 1790395375588331 us; error 31 us; skew 500 ppm
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:55.591151 1702 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:02:55.591617 1702 webserver.cc:460] Webserver started at http://127.0.200.131:37023/ using document root <none> and password file <none>
I20260926 04:02:55.591867 1702 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:55.591938 1702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:55.592084 1702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:55.593248 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "7a2ec2f54df24912adad57a5261f12ab"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "6efb2ee8ff50745ebeebf7b3ad55d0e7"
server_key_iv: "9932409e4bfafebb68e3b983eb217e34"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.593622 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "7a2ec2f54df24912adad57a5261f12ab"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "6efb2ee8ff50745ebeebf7b3ad55d0e7"
server_key_iv: "9932409e4bfafebb68e3b983eb217e34"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.595137 1702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:02:55.595957 1717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.596282 1702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:55.596408 1702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "7a2ec2f54df24912adad57a5261f12ab"
format_stamp: "Formatted at 2026-09-26 04:02:55 on dist-test-slave-j8c1"
server_key: "6efb2ee8ff50745ebeebf7b3ad55d0e7"
server_key_iv: "9932409e4bfafebb68e3b983eb217e34"
server_key_version: "encryptionkey@0"
I20260926 04:02:55.596491 1702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:55.615454 1702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:55.617928 1702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:55.618116 1702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:55.618372 1702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:02:55.618783 1702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:55.618839 1702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.618881 1702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:55.618911 1702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:55.624663 1702 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:36707
I20260926 04:02:55.624706 1830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:36707 every 8 connection(s)
I20260926 04:02:55.625082 1702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:02:55.627229 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1702
I20260926 04:02:55.627334 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:02:55.627473 802 external_mini_cluster.cc:1428] Setting key 44d104c2d57a5e7494c1dd99877ffacd
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:55.635385 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.625895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:41011 (local address 127.0.200.190:42425)
0926 04:02:55.625989 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.625991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.626273 (+ 282us) server_negotiation.cc:407] Connection header received
0926 04:02:55.626910 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.626912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.626924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.626983 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.628246 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.628845 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.629939 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.630123 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.630185 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.632567 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.632569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.632569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.632576 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.634006 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.634414 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.634415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.634416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.634475 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.634757 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.634758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.634758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.634876 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.634933 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.635218 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.635306 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 04:02:55.635494 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.625894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42425 (local address 127.0.200.131:41011)
0926 04:02:55.626140 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.626180 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.626805 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.627062 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.627065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.627793 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:55.628090 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.628096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.628934 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.628935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.629857 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.629859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.630223 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.630224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.630343 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.630998 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:55.631015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:55.632469 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:55.634078 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.634084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.634086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.634335 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.634539 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:55.634539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:55.634540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:55.634696 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:55.634933 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:55.634937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:55.635059 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.635141 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.635299 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260926 04:02:55.636013 1831 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42425
I20260926 04:02:55.636122 1831 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:55.636312 1831 heartbeater.cc:511] Master 127.0.200.190:42425 requested a full tablet report, sending...
I20260926 04:02:55.636700 1372 ts_manager.cc:194] Registered new tserver with Master: 7a2ec2f54df24912adad57a5261f12ab (127.0.200.131:36707)
I20260926 04:02:55.637063 1372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:41011
I20260926 04:02:55.640085 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:02:55.648103 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.642368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.0.200.190:42425)
0926 04:02:55.642413 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.642415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.642515 (+ 100us) server_negotiation.cc:407] Connection header received
0926 04:02:55.642628 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.642629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.642640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.642696 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.643282 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.643768 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.644504 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.644679 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.644732 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.645836 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.645837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.645838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.645843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.647222 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.647513 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.647513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.647513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.647555 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.647693 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.647693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.647694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.647833 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.647905 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.647982 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.648039 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:02:55.650385 1372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:02:55.651947 1372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:02:55.658337 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:35599 (local address 127.0.0.1:49240)
0926 04:02:55.654499 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.654509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.654570 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.654930 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.654931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.654933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:55.655006 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.655008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.655957 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.655958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.657295 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.657298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.658022 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.658098 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658112 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.658180 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.658235 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":33,"thread_start_us":61,"threads_started":1}
I20260926 04:02:55.658406 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.0.200.130:40719)
0926 04:02:55.654756 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.654760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.654769 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:02:55.654817 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.654819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.654835 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.654912 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:55.655167 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.656274 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.657340 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.658073 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658159 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658226 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.658284 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.658323 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":81,"threads_started":1}
I20260926 04:02:55.658479 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:36707 (local address 127.0.0.1:55550)
0926 04:02:55.654313 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.654355 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.654427 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.654842 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.654845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.654847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:55.654916 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.654917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.656335 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.656336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.657401 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.657402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.658250 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.658323 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658328 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.658358 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.658397 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260926 04:02:55.658740 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.0.200.129:35599)
0926 04:02:55.654753 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.654757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.654768 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:02:55.654811 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.654813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.654827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.654927 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:55.655086 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.655856 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.657393 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.657935 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658023 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658084 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.658613 (+ 529us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.658666 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260926 04:02:55.658897 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40719 (local address 127.0.0.1:50804)
0926 04:02:55.654471 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.654482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.654534 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.654991 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.654992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.654993 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:55.655024 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.655025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.656376 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.656377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.657263 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.657265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.658159 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.658221 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658224 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.658274 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.658306 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":80,"threads_started":1}
I20260926 04:02:55.658798 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.654333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.0.200.131:36707)
0926 04:02:55.654648 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.654657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.654681 (+ 24us) server_negotiation.cc:407] Connection header received
0926 04:02:55.654722 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.654725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.654741 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.654827 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:55.655000 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.656230 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.657483 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.658158 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.658579 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.658643 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.658665 (+ 22us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.658715 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":86,"threads_started":1}
I20260926 04:02:55.659473 1497 tablet_service.cc:1467] Processing CreateTablet for tablet ed04ec9d2bf64beb9395b6a73784af78 (DEFAULT_TABLE table=test-table [id=e6bd75bbeac1481eaf97cfdc1c1657c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:55.659843 1497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed04ec9d2bf64beb9395b6a73784af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:55.659823 1631 tablet_service.cc:1467] Processing CreateTablet for tablet ed04ec9d2bf64beb9395b6a73784af78 (DEFAULT_TABLE table=test-table [id=e6bd75bbeac1481eaf97cfdc1c1657c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:55.659809 1765 tablet_service.cc:1467] Processing CreateTablet for tablet ed04ec9d2bf64beb9395b6a73784af78 (DEFAULT_TABLE table=test-table [id=e6bd75bbeac1481eaf97cfdc1c1657c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:55.660140 1765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed04ec9d2bf64beb9395b6a73784af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:55.660148 1631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed04ec9d2bf64beb9395b6a73784af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:55.662801 1848 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Bootstrap starting.
I20260926 04:02:55.663450 1848 tablet_bootstrap.cc:654] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:55.663434 1849 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Bootstrap starting.
I20260926 04:02:55.664126 1848 log.cc:824] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:55.664173 1849 tablet_bootstrap.cc:654] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:55.664723 1849 log.cc:824] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:55.664965 1848 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: No bootstrap required, opened a new log
I20260926 04:02:55.665040 1848 ts_tablet_manager.cc:1397] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:02:55.665079 1850 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Bootstrap starting.
I20260926 04:02:55.665503 1849 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: No bootstrap required, opened a new log
I20260926 04:02:55.665592 1849 ts_tablet_manager.cc:1397] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:02:55.665755 1850 tablet_bootstrap.cc:654] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:55.666289 1850 log.cc:824] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:55.667119 1850 tablet_bootstrap.cc:492] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: No bootstrap required, opened a new log
I20260926 04:02:55.667210 1850 ts_tablet_manager.cc:1397] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:02:55.667394 1848 raft_consensus.cc:348] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.667562 1848 raft_consensus.cc:374] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:55.667605 1848 raft_consensus.cc:729] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50277a38174d41558f6ef031aa5eda1c, State: Initialized, Role: FOLLOWER
I20260926 04:02:55.667769 1848 consensus_queue.cc:260] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.668013 1849 raft_consensus.cc:348] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.668193 1848 ts_tablet_manager.cc:1428] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:02:55.668211 1849 raft_consensus.cc:374] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:55.668267 1849 raft_consensus.cc:729] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867bb6c8fa2f49baa88721c434e039d7, State: Initialized, Role: FOLLOWER
I20260926 04:02:55.668349 1563 heartbeater.cc:503] Master 127.0.200.190:42425 was elected leader, sending a full tablet report...
I20260926 04:02:55.668406 1849 consensus_queue.cc:260] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.668706 1849 ts_tablet_manager.cc:1428] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:02:55.668826 1697 heartbeater.cc:503] Master 127.0.200.190:42425 was elected leader, sending a full tablet report...
I20260926 04:02:55.669890 1850 raft_consensus.cc:348] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.670058 1850 raft_consensus.cc:374] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:55.670112 1850 raft_consensus.cc:729] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2ec2f54df24912adad57a5261f12ab, State: Initialized, Role: FOLLOWER
I20260926 04:02:55.670221 1850 consensus_queue.cc:260] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.670493 1850 ts_tablet_manager.cc:1428] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 04:02:55.670493 1831 heartbeater.cc:503] Master 127.0.200.190:42425 was elected leader, sending a full tablet report...
W20260926 04:02:55.711560 1698 tablet.cc:2369] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:02:55.810016 1564 tablet.cc:2369] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:02:55.875692 1832 tablet.cc:2369] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:02:55.909437 1856 raft_consensus.cc:482] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:02:55.909569 1856 raft_consensus.cc:504] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.909946 1856 leader_election.cc:288] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50277a38174d41558f6ef031aa5eda1c (127.0.200.129:35599), 867bb6c8fa2f49baa88721c434e039d7 (127.0.200.130:40719)
I20260926 04:02:55.914125 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.910354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:39569 (local address 127.0.200.130:40719)
0926 04:02:55.910400 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.910402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.910520 (+ 118us) server_negotiation.cc:407] Connection header received
0926 04:02:55.910567 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.910568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.910579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.910636 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:55.910781 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.911558 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913130 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.913738 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913799 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.913806 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.914003 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.914044 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:02:55.914125 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.910279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:57001 (local address 127.0.200.129:35599)
0926 04:02:55.910355 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.910357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.910368 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:02:55.910463 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.910464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.910475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.910586 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:55.910781 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.911529 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913130 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.913689 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913791 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.913802 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.914030 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.914065 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 04:02:55.914196 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.910164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:35599 (local address 127.0.200.131:57001)
0926 04:02:55.910311 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.910325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.910383 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.910589 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.910590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.910592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:55.910680 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.910683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.911673 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.911675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.913014 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.913018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.913823 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.913919 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.913921 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.914019 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.914054 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":78}
I20260926 04:02:55.914203 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.910207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40719 (local address 127.0.200.131:39569)
0926 04:02:55.910440 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:55.910449 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:55.910503 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:55.910649 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:55.910650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:55.910651 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:55.910707 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.910708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.911673 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.911675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.913014 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:55.913018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.913824 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.913825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:55.913916 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.913920 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:02:55.913988 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:02:55.914053 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":41,"thread_start_us":171,"threads_started":1}
I20260926 04:02:55.914587 1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed04ec9d2bf64beb9395b6a73784af78" candidate_uuid: "7a2ec2f54df24912adad57a5261f12ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50277a38174d41558f6ef031aa5eda1c" is_pre_election: true
I20260926 04:02:55.914770 1517 raft_consensus.cc:2454] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a2ec2f54df24912adad57a5261f12ab in term 0.
I20260926 04:02:55.915067 1720 leader_election.cc:302] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50277a38174d41558f6ef031aa5eda1c, 7a2ec2f54df24912adad57a5261f12ab; no voters:
I20260926 04:02:55.915063 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed04ec9d2bf64beb9395b6a73784af78" candidate_uuid: "7a2ec2f54df24912adad57a5261f12ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867bb6c8fa2f49baa88721c434e039d7" is_pre_election: true
I20260926 04:02:55.915197 1651 raft_consensus.cc:2454] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a2ec2f54df24912adad57a5261f12ab in term 0.
I20260926 04:02:55.915210 1856 raft_consensus.cc:2781] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:02:55.915252 1856 raft_consensus.cc:482] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:02:55.915285 1856 raft_consensus.cc:3037] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:55.916045 1856 raft_consensus.cc:504] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.916193 1856 leader_election.cc:288] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [CANDIDATE]: Term 1 election: Requested vote from peers 50277a38174d41558f6ef031aa5eda1c (127.0.200.129:35599), 867bb6c8fa2f49baa88721c434e039d7 (127.0.200.130:40719)
I20260926 04:02:55.916359 1651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed04ec9d2bf64beb9395b6a73784af78" candidate_uuid: "7a2ec2f54df24912adad57a5261f12ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867bb6c8fa2f49baa88721c434e039d7"
I20260926 04:02:55.916363 1517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed04ec9d2bf64beb9395b6a73784af78" candidate_uuid: "7a2ec2f54df24912adad57a5261f12ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50277a38174d41558f6ef031aa5eda1c"
I20260926 04:02:55.916435 1651 raft_consensus.cc:3037] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:55.916456 1517 raft_consensus.cc:3037] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:55.917140 1651 raft_consensus.cc:2454] T ed04ec9d2bf64beb9395b6a73784af78 P 867bb6c8fa2f49baa88721c434e039d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2ec2f54df24912adad57a5261f12ab in term 1.
I20260926 04:02:55.917147 1517 raft_consensus.cc:2454] T ed04ec9d2bf64beb9395b6a73784af78 P 50277a38174d41558f6ef031aa5eda1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a2ec2f54df24912adad57a5261f12ab in term 1.
I20260926 04:02:55.917309 1720 leader_election.cc:302] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50277a38174d41558f6ef031aa5eda1c, 7a2ec2f54df24912adad57a5261f12ab; no voters:
I20260926 04:02:55.917418 1856 raft_consensus.cc:2781] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:55.917584 1856 raft_consensus.cc:686] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [term 1 LEADER]: Becoming Leader. State: Replica: 7a2ec2f54df24912adad57a5261f12ab, State: Running, Role: LEADER
I20260926 04:02:55.917685 1856 consensus_queue.cc:237] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } }
I20260926 04:02:55.918512 1372 catalog_manager.cc:5495] T ed04ec9d2bf64beb9395b6a73784af78 P 7a2ec2f54df24912adad57a5261f12ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a2ec2f54df24912adad57a5261f12ab (127.0.200.131). New cstate: current_term: 1 leader_uuid: "7a2ec2f54df24912adad57a5261f12ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a2ec2f54df24912adad57a5261f12ab" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50277a38174d41558f6ef031aa5eda1c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 35599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "867bb6c8fa2f49baa88721c434e039d7" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40719 } health_report { overall_health: UNKNOWN } } }
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:55.946831 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.938430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.0.200.129:35599)
0926 04:02:55.938493 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.938495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.938628 (+ 133us) server_negotiation.cc:407] Connection header received
0926 04:02:55.938700 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.938701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.938710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.938764 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.939371 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.939920 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.940648 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.940880 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.940939 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.943584 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.943595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.943595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.943602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.945355 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.945701 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.945704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.945705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.945766 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.946062 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.946063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.946064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.946236 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.946373 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.946528 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.946682 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:55.971201 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.963650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.0.200.129:35599)
0926 04:02:55.963734 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.963737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.963812 (+ 75us) server_negotiation.cc:407] Connection header received
0926 04:02:55.963932 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.963933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.963944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.964002 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.964612 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.965105 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.965812 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.965997 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.966056 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.968543 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.968545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.968545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.968554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.969946 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.970319 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.970320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.970321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.970379 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.970611 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.970612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.970613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.970723 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.970819 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.971012 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.971095 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 04:02:55 dist-test-slave-j8c1 krb5kdc[1327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:55.995374 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:55.987642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.0.200.129:35599)
0926 04:02:55.987712 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:55.987715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:55.987845 (+ 130us) server_negotiation.cc:407] Connection header received
0926 04:02:55.987929 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:55.987930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:55.987940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:55.987992 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:55.988814 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.989302 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.990485 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:55.990730 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:55.990794 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:55.993123 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:55.993124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:55.993124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:55.993131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:55.994350 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.994735 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.994735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.994736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.994787 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:55.994948 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:55.994948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:55.994948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:55.995032 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:55.995101 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:55.995248 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 04:02:55.995305 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 04:02:55.995858 1477 tablet_service.cc:2908] Rejecting scan request for tablet ed04ec9d2bf64beb9395b6a73784af78: Uninitialized: clean time has not yet been initialized
I20260926 04:02:55.996759 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1434
I20260926 04:02:56.001907 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1568
I20260926 04:02:56.007473 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1702
I20260926 04:02:56.011948 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1340
2026-09-26T04:02:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2248 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:02:56 dist-test-slave-j8c1 krb5kdc[1872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:02:56 dist-test-slave-j8c1 krb5kdc[1872](info): set up 2 sockets
Sep 26 04:02:56 dist-test-slave-j8c1 krb5kdc[1872](info): commencing operation
krb5kdc: starting...
W20260926 04:02:57.062281 802 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:02:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:02:57.186211 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40817
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:40817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:57.273227 1885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:57.273474 1885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:57.273500 1885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:57.274762 1885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:02:57.274808 1885 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 04:02:57.274822 1885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:57.274837 1885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:57.274850 1885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:02:57.274863 1885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:02:57.276422 1885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:40817
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:57.276638 1885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:57.276825 1885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:57.282100 1893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.282117 1890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.282100 1891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:57.282691 1885 server_base.cc:1034] running on GCE node
I20260926 04:02:57.282975 1885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:57.283291 1885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:57.284456 1885 hybrid_clock.cc:648] HybridClock initialized: now 1790395377284436 us; error 41 us; skew 500 ppm
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:57.293128 1885 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:02:57.293684 1885 webserver.cc:460] Webserver started at http://127.0.200.190:45825/ using document root <none> and password file <none>
I20260926 04:02:57.293941 1885 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:57.294010 1885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:57.294154 1885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:57.295327 1885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "a9e04755f4c14579a23f878915ed8e4f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e8599e9dc2459a6168c37f3eeba5da0c"
server_key_iv: "f3c0148797078ee5c77b8db2a7d72721"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.295699 1885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "a9e04755f4c14579a23f878915ed8e4f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e8599e9dc2459a6168c37f3eeba5da0c"
server_key_iv: "f3c0148797078ee5c77b8db2a7d72721"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.297051 1885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:02:57.297796 1900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.298000 1885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:02:57.298106 1885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "a9e04755f4c14579a23f878915ed8e4f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e8599e9dc2459a6168c37f3eeba5da0c"
server_key_iv: "f3c0148797078ee5c77b8db2a7d72721"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.298180 1885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:57.313531 1885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:57.315955 1885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:57.316129 1885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:57.321265 1885 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:40817
I20260926 04:02:57.321420 1962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:40817 every 8 connection(s)
I20260926 04:02:57.321736 1885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:02:57.322420 1963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.324803 1963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Bootstrap starting.
I20260926 04:02:57.325438 1963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.325729 1963 log.cc:824] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:57.326488 1963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: No bootstrap required, opened a new log
I20260926 04:02:57.328660 1963 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } }
I20260926 04:02:57.328816 1963 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.328847 1963 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e04755f4c14579a23f878915ed8e4f, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.329003 1963 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } }
I20260926 04:02:57.329087 1963 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:02:57.329140 1963 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:02:57.329195 1963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:57.329823 1963 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } }
I20260926 04:02:57.329918 1963 leader_election.cc:302] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9e04755f4c14579a23f878915ed8e4f; no voters:
I20260926 04:02:57.330111 1963 leader_election.cc:288] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:02:57.330154 1966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:57.330426 1963 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:02:57.330430 1966 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [term 1 LEADER]: Becoming Leader. State: Replica: a9e04755f4c14579a23f878915ed8e4f, State: Running, Role: LEADER
I20260926 04:02:57.330677 1966 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } }
I20260926 04:02:57.331096 1968 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9e04755f4c14579a23f878915ed8e4f. Latest consensus state: current_term: 1 leader_uuid: "a9e04755f4c14579a23f878915ed8e4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } } }
I20260926 04:02:57.331113 1967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9e04755f4c14579a23f878915ed8e4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e04755f4c14579a23f878915ed8e4f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40817 } } }
I20260926 04:02:57.331198 1968 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [sys.catalog]: This master's current role is: LEADER
I20260926 04:02:57.331249 1967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f [sys.catalog]: This master's current role is: LEADER
I20260926 04:02:57.331452 1975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:02:57.331812 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1885
I20260926 04:02:57.331904 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:02:57.332027 1975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:02:57.332072 802 external_mini_cluster.cc:1428] Setting key c273b4b7e86fb04b42e95514c18ff026
I20260926 04:02:57.333798 1975 catalog_manager.cc:1347] Generated new cluster ID: d9828a43288b4edeb7f608cb62f80dac
I20260926 04:02:57.333851 1975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:57.340433 1975 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:02:57.341125 1975 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:02:57.342941 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.332779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.0.200.190:40817)
0926 04:02:57.332957 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.332967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.333003 (+ 36us) server_negotiation.cc:407] Connection header received
0926 04:02:57.333314 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.333321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.333403 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.333556 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.334244 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.335259 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.336116 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.336597 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.336724 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.339214 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.339219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.339219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.339247 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.341303 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.341691 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.341694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.341695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.341795 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.342003 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.342004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.342005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.342239 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.342418 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.342492 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.342700 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260926 04:02:57.348150 1975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Generated new TSK 0
I20260926 04:02:57.348398 1975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:02:57.378794 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40817
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:57.462113 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:57.462324 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:57.462350 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:57.463840 1989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:57.463894 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:57.463918 1989 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:02:57.463937 1989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:02:57.465616 1989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:57.465879 1989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:57.466122 1989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:57.472261 1994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.472288 1995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.472355 1997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:57.472640 1989 server_base.cc:1034] running on GCE node
I20260926 04:02:57.472864 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:57.473136 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:57.474300 1989 hybrid_clock.cc:648] HybridClock initialized: now 1790395377474283 us; error 31 us; skew 500 ppm
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:57.476857 1989 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:02:57.477326 1989 webserver.cc:460] Webserver started at http://127.0.200.129:43383/ using document root <none> and password file <none>
I20260926 04:02:57.477523 1989 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:57.477564 1989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:57.477656 1989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:57.478760 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "bb2ee886a2f54312bd82d53938525fa7"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "d58ce7c2bba66e03a79895eb4d7fdc09"
server_key_iv: "543ba3dba8db9e194f833158a3769f97"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.479079 1989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "bb2ee886a2f54312bd82d53938525fa7"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "d58ce7c2bba66e03a79895eb4d7fdc09"
server_key_iv: "543ba3dba8db9e194f833158a3769f97"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.480477 1989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:02:57.481264 2004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.481451 1989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:57.481540 1989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "bb2ee886a2f54312bd82d53938525fa7"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "d58ce7c2bba66e03a79895eb4d7fdc09"
server_key_iv: "543ba3dba8db9e194f833158a3769f97"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.481606 1989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:57.505381 1989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:57.507797 1989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:57.507928 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:57.508405 1989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:57.508451 1989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.508476 1989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:57.508488 1989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.515733 1989 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:43279
I20260926 04:02:57.515796 2119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:43279 every 8 connection(s)
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:57.516168 1989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:02:57.519577 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.509024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:48015 (local address 127.0.200.190:40817)
0926 04:02:57.509100 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.509103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.509255 (+ 152us) server_negotiation.cc:407] Connection header received
0926 04:02:57.509766 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.509767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.509784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.509867 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.510797 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.511651 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.512773 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.513173 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.513264 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.516665 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.516667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.516667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.516676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.518141 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.518587 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.518588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.518589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.518650 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.518924 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.518925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.518925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.519043 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.519116 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.519397 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.519482 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:02:57.519759 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.508966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.129:48015)
0926 04:02:57.509119 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.509161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.509661 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.509941 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.509945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.510361 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.510681 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.510689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.511754 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.511756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.512671 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.512673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.513281 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.513283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.513441 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.514634 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.514647 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.516542 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.518255 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.518261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.518264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.518489 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.518732 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.518733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.518733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.518851 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.519106 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.519112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.519260 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.519363 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.519528 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260926 04:02:57.520298 2120 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40817
I20260926 04:02:57.520390 2120 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:57.520619 2120 heartbeater.cc:511] Master 127.0.200.190:40817 requested a full tablet report, sending...
I20260926 04:02:57.521183 1916 ts_manager.cc:194] Registered new tserver with Master: bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279)
I20260926 04:02:57.521838 1916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:48015
I20260926 04:02:57.524236 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 1989
I20260926 04:02:57.524360 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:02:57.524538 802 external_mini_cluster.cc:1428] Setting key ffa6cde8918c44298db2bfc16755f623
I20260926 04:02:57.527346 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.521338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.129:33101)
0926 04:02:57.521482 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.521491 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.521559 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.521695 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.521695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.521874 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.521922 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.521923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.522731 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.522732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.523283 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.523285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.523662 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.523663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.523736 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.524212 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.524222 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.524614 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.526203 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.526204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.526205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.526381 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.526629 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.526630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.526630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.526707 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.526998 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.526999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.527044 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.527112 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.527204 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260926 04:02:57.527354 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.521382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:33101 (local address 127.0.200.190:40817)
0926 04:02:57.521428 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.521431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.521545 (+ 114us) server_negotiation.cc:407] Connection header received
0926 04:02:57.521623 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.521624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.521635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.521690 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.522072 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.522626 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.523378 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.523584 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.523653 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.524694 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.524696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.524696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.524704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.526115 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.526470 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.526472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.526472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.526533 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.526776 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.526777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.526777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.526917 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.526990 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.527152 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.527234 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:02:57.554462 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40817
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:57.641680 2129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:57.641909 2129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:57.641934 2129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:57.643107 2129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:57.643147 2129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:57.643168 2129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:02:57.643186 2129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:02:57.644675 2129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:57.644876 2129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:57.645062 2129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:57.651114 2135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.651108 2134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.651111 2137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:57.651405 2129 server_base.cc:1034] running on GCE node
I20260926 04:02:57.651692 2129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:57.652004 2129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:57.653218 2129 hybrid_clock.cc:648] HybridClock initialized: now 1790395377653209 us; error 44 us; skew 500 ppm
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:57.656075 2129 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:02:57.656628 2129 webserver.cc:460] Webserver started at http://127.0.200.130:34155/ using document root <none> and password file <none>
I20260926 04:02:57.656852 2129 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:57.656916 2129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:57.657063 2129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:57.658140 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e54044a40bacd060f7d51e707bd326d9"
server_key_iv: "715e6f9ecbd1ff2f9c6677e01bb040d5"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.658529 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e54044a40bacd060f7d51e707bd326d9"
server_key_iv: "715e6f9ecbd1ff2f9c6677e01bb040d5"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.659893 2129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:57.660645 2144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.660823 2129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:02:57.660923 2129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "e54044a40bacd060f7d51e707bd326d9"
server_key_iv: "715e6f9ecbd1ff2f9c6677e01bb040d5"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.661006 2129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:57.668088 2129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:57.670967 2129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:57.671172 2129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:57.671835 2129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:57.671888 2129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.671924 2129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:57.671957 2129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:57.679292 2129 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:35075
I20260926 04:02:57.679358 2259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:35075 every 8 connection(s)
I20260926 04:02:57.679721 2129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:02:57.682214 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.672254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44923 (local address 127.0.200.190:40817)
0926 04:02:57.672324 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.672327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.672557 (+ 230us) server_negotiation.cc:407] Connection header received
0926 04:02:57.673067 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.673069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.673084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.673176 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.673856 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.674701 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.675751 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.675971 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.676058 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.679143 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.679146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.679146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.679154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.680769 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.681236 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.681237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.681238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.681310 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.681567 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.681568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.681569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.681693 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.681768 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.682037 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.682121 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 04:02:57.682399 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.672253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.130:44923)
0926 04:02:57.672420 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.672462 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.672953 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.673230 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.673234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.673559 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.673763 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.673767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.674798 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.674799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.675661 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.675664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.676071 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.676072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.676203 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.676774 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.676782 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.679015 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.680898 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.680904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.680907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.681134 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.681376 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.681377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.681378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.681491 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.681758 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.681764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.681902 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.682009 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.682168 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260926 04:02:57.682891 2260 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40817
I20260926 04:02:57.682973 2260 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:57.683182 2260 heartbeater.cc:511] Master 127.0.200.190:40817 requested a full tablet report, sending...
I20260926 04:02:57.683583 1916 ts_manager.cc:194] Registered new tserver with Master: e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075)
I20260926 04:02:57.684082 1916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:44923
I20260926 04:02:57.689913 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2129
I20260926 04:02:57.690040 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:02:57.690215 802 external_mini_cluster.cc:1428] Setting key cf6a6e8e2186fa4addff345a51f90cf3
I20260926 04:02:57.690213 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.683973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43467 (local address 127.0.200.190:40817)
0926 04:02:57.684017 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.684019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.684097 (+ 78us) server_negotiation.cc:407] Connection header received
0926 04:02:57.684193 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.684194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.684205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.684258 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.684654 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.685216 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.686056 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.686237 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.686311 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.687253 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.687254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.687254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.687260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.688766 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.689137 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.689138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.689138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.689195 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.689510 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.689537 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.689537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.689757 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.689843 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.690031 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.690115 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:02:57.690231 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.683894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.130:43467)
0926 04:02:57.684036 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.684046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.684109 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.684261 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.684261 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.684483 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.684573 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.684574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.685300 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.685301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.685954 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.685956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.686310 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.686311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.686356 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.686839 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.686844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.687175 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.688867 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.688868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.688868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.689049 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.689283 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.689284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.689285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.689352 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.689857 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.689858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.689909 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.689979 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.690130 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:02:57.717676 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40817
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:57.803670 2269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:57.803925 2269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:57.803967 2269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:57.805260 2269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:57.805338 2269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:57.805377 2269 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:02:57.805419 2269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:02:57.807128 2269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:57.807446 2269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:57.807691 2269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:57.813215 2275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.813261 2277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:57.813247 2274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:57.813439 2269 server_base.cc:1034] running on GCE node
I20260926 04:02:57.813753 2269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:57.814046 2269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:57.815224 2269 hybrid_clock.cc:648] HybridClock initialized: now 1790395377815195 us; error 47 us; skew 500 ppm
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:57.817936 2269 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:02:57.818454 2269 webserver.cc:460] Webserver started at http://127.0.200.131:37807/ using document root <none> and password file <none>
I20260926 04:02:57.818745 2269 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:57.818813 2269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:57.818962 2269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:57.820130 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "ce683b3b7f734d61a93b2e4155657fa4"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "838b2e1c38ccc604ea3359cc6bb9755b"
server_key_iv: "2a41077a3acb9f46da86acfe88464c6d"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.820541 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "ce683b3b7f734d61a93b2e4155657fa4"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "838b2e1c38ccc604ea3359cc6bb9755b"
server_key_iv: "2a41077a3acb9f46da86acfe88464c6d"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.822103 2269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:02:57.823127 2284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.823347 2269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:02:57.823462 2269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "ce683b3b7f734d61a93b2e4155657fa4"
format_stamp: "Formatted at 2026-09-26 04:02:57 on dist-test-slave-j8c1"
server_key: "838b2e1c38ccc604ea3359cc6bb9755b"
server_key_iv: "2a41077a3acb9f46da86acfe88464c6d"
server_key_version: "encryptionkey@0"
I20260926 04:02:57.823562 2269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:57.839216 2269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:57.841637 2269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:57.841818 2269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:57.842375 2269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:02:57.842440 2269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.842496 2269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:02:57.842526 2269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:57.850116 2269 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:37409
I20260926 04:02:57.850179 2399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:37409 every 8 connection(s)
I20260926 04:02:57.850613 2269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:02:57.852628 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.842955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:45705 (local address 127.0.200.190:40817)
0926 04:02:57.843005 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.843008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.843237 (+ 229us) server_negotiation.cc:407] Connection header received
0926 04:02:57.843783 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.843785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.843802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.843881 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.844613 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.845464 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.846663 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.846861 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.846941 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.849422 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.849423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.849423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.849432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.851192 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.851684 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.851685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.851685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.851744 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.851971 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.851971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.851972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.852092 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.852151 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.852473 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.852556 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:02:57.852803 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.842942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.131:45705)
0926 04:02:57.843094 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.843145 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.843674 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.844010 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.844015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.844283 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.844519 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.844523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.845572 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.845574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.846529 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.846532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.846959 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.846960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.847114 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.847679 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.847689 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.849319 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.851307 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.851313 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.851315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.851590 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.851815 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.851816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.851816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.851911 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.852183 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.852190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.852317 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.852425 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.852571 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260926 04:02:57.853312 2400 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40817
I20260926 04:02:57.853399 2400 heartbeater.cc:464] Registering TS with master...
I20260926 04:02:57.853498 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2269
I20260926 04:02:57.853593 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:02:57.853628 2400 heartbeater.cc:511] Master 127.0.200.190:40817 requested a full tablet report, sending...
I20260926 04:02:57.853794 802 external_mini_cluster.cc:1428] Setting key a9a1043612e6ec2ec01973e641935f71
I20260926 04:02:57.854043 1917 ts_manager.cc:194] Registered new tserver with Master: ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131:37409)
I20260926 04:02:57.854609 1917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:45705
I20260926 04:02:57.855057 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:02:57.860988 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.854511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:49345 (local address 127.0.200.190:40817)
0926 04:02:57.854580 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.854582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.854724 (+ 142us) server_negotiation.cc:407] Connection header received
0926 04:02:57.854828 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.854829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.854840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.854889 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.855297 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.855947 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.856879 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.857103 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.857164 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.858320 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.858322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.858322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.858328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.859791 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.860169 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.860171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.860171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.860228 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.860421 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.860422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.860422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.860536 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.860609 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.860810 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.860905 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:02:57.861035 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.854483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.200.131:49345)
0926 04:02:57.854650 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.854662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.854751 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.854892 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.854892 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.855126 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:57.855204 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.855207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.856059 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.856060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.856767 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.856770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.857182 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.857184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.857263 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.857759 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:57.857765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:57.858219 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:57.859897 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.859898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.859898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.860084 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.860292 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:57.860293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:57.860293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:57.860354 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:57.860600 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:57.860601 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:57.860651 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.860743 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.860878 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 04:02:57 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:57.890642 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.881464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.0.200.190:40817)
0926 04:02:57.881536 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.881539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.881549 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:02:57.881608 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.881609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.881623 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.881751 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:57.882676 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.883520 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.884670 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.884971 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.885043 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.887872 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:57.887873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:57.887873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:57.887881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:57.889343 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.889800 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.889801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.889801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.889865 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:57.890113 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:57.890114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:57.890115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:57.890240 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:57.890340 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.890432 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.890496 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:02:57.892671 1917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:02:57.894542 1917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:02:57.901571 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.0.200.130:35075)
0926 04:02:57.897626 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.897632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.897643 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:02:57.897692 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.897694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.897711 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.897996 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:57.898052 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.898873 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.900313 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.901107 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.901184 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.901261 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.901398 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.901440 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260926 04:02:57.901562 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35075 (local address 127.0.0.1:45454)
0926 04:02:57.897329 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.897340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.897452 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.897828 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.897832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.897834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:57.897917 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.897919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.898959 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.898960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.900223 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.900226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.901186 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.901187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.901268 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.901283 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.901359 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.901456 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260926 04:02:57.901674 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:37409 (local address 127.0.0.1:52284)
0926 04:02:57.897276 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.897323 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.897401 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.897994 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.897995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.897997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:57.898028 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.898029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.899329 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.899330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.900492 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.900494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.901407 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.901408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.901471 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.901475 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.901527 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.901557 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":28,"thread_start_us":68,"threads_started":1}
I20260926 04:02:57.901747 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.0.200.131:37409)
0926 04:02:57.897751 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.897755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.897764 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:02:57.897800 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.897802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.897817 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.897885 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:57.898095 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.899229 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.900574 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.901310 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.901400 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.901468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.901537 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.901578 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":100,"threads_started":1}
I20260926 04:02:57.902382 2194 tablet_service.cc:1467] Processing CreateTablet for tablet fc4ec3b7827645478e63e78f02d1b24e (DEFAULT_TABLE table=test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.902588 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43279 (local address 127.0.0.1:45726)
0926 04:02:57.897582 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.897593 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.897637 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.897927 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.897928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.897930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:57.897997 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.897999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.899334 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.899335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.900307 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.900310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.902296 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.902297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.902371 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.902376 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.902448 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.902489 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":30,"thread_start_us":68,"threads_started":1}
I20260926 04:02:57.902464 2334 tablet_service.cc:1467] Processing CreateTablet for tablet fc4ec3b7827645478e63e78f02d1b24e (DEFAULT_TABLE table=test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.902817 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.897572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.0.200.129:43279)
0926 04:02:57.897704 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.897708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.897719 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:02:57.897757 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.897759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.897775 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.897859 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:57.898076 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.899249 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.900391 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.901108 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.901196 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.901273 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.902664 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.902710 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260926 04:02:57.902842 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4ec3b7827645478e63e78f02d1b24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.902843 2194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4ec3b7827645478e63e78f02d1b24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.905969 2423 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Bootstrap starting.
I20260926 04:02:57.906009 2424 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Bootstrap starting.
I20260926 04:02:57.906695 2423 tablet_bootstrap.cc:654] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.906531 2054 tablet_service.cc:1467] Processing CreateTablet for tablet fc4ec3b7827645478e63e78f02d1b24e (DEFAULT_TABLE table=test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.906760 2424 tablet_bootstrap.cc:654] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.906972 2054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc4ec3b7827645478e63e78f02d1b24e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.907038 2423 log.cc:824] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:57.907068 2424 log.cc:824] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:57.907876 2424 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: No bootstrap required, opened a new log
I20260926 04:02:57.907960 2424 ts_tablet_manager.cc:1397] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:02:57.910158 2424 raft_consensus.cc:348] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.910351 2424 raft_consensus.cc:374] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.910390 2424 raft_consensus.cc:729] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84fe4e7683b46fbb66d0e38d2b5457f, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.910527 2424 consensus_queue.cc:260] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.910851 2424 ts_tablet_manager.cc:1428] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:02:57.911029 2260 heartbeater.cc:503] Master 127.0.200.190:40817 was elected leader, sending a full tablet report...
I20260926 04:02:57.911026 2428 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Bootstrap starting.
I20260926 04:02:57.911433 2423 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: No bootstrap required, opened a new log
I20260926 04:02:57.911531 2423 ts_tablet_manager.cc:1397] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260926 04:02:57.911832 2428 tablet_bootstrap.cc:654] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.912184 2428 log.cc:824] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Log is configured to *not* fsync() on all Append() calls
I20260926 04:02:57.913012 2428 tablet_bootstrap.cc:492] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: No bootstrap required, opened a new log
I20260926 04:02:57.913106 2428 ts_tablet_manager.cc:1397] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:02:57.913938 2423 raft_consensus.cc:348] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.914145 2423 raft_consensus.cc:374] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.914186 2423 raft_consensus.cc:729] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce683b3b7f734d61a93b2e4155657fa4, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.914305 2423 consensus_queue.cc:260] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.914803 2423 ts_tablet_manager.cc:1428] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260926 04:02:57.914827 2400 heartbeater.cc:503] Master 127.0.200.190:40817 was elected leader, sending a full tablet report...
I20260926 04:02:57.915791 2428 raft_consensus.cc:348] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.915971 2428 raft_consensus.cc:374] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.916023 2428 raft_consensus.cc:729] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2ee886a2f54312bd82d53938525fa7, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.916189 2428 consensus_queue.cc:260] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.916474 2428 ts_tablet_manager.cc:1428] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 04:02:57.916508 2120 heartbeater.cc:503] Master 127.0.200.190:40817 was elected leader, sending a full tablet report...
W20260926 04:02:57.930733 2261 tablet.cc:2369] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:02:57.949750 2431 raft_consensus.cc:482] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:02:57.949884 2431 raft_consensus.cc:504] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.950276 2431 leader_election.cc:288] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075), ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131:37409)
I20260926 04:02:57.954505 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.950644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:33365 (local address 127.0.200.131:37409)
0926 04:02:57.950691 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.950693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.950885 (+ 192us) server_negotiation.cc:407] Connection header received
0926 04:02:57.950909 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.950910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.950917 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.950963 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:57.951112 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.951790 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.953405 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.954096 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.954159 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.954167 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.954385 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.954429 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:02:57.954576 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.950622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44267 (local address 127.0.200.130:35075)
0926 04:02:57.950668 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.950671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.950709 (+ 38us) server_negotiation.cc:407] Connection header received
0926 04:02:57.950738 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.950739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.950750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.950804 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:57.951010 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.951777 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.953402 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.954062 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.954149 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.954160 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.954386 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.954429 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:02:57.954524 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.950501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35075 (local address 127.0.200.129:44267)
0926 04:02:57.950591 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.950603 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.950659 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.950840 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.950840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.950843 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:57.950920 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.950925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.951905 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.951906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.953273 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.953277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.954187 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.954189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.954292 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.954298 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.954368 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.954417 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":58}
I20260926 04:02:57.954900 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.950502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:37409 (local address 127.0.200.129:33365)
0926 04:02:57.950765 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:57.950772 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:57.950848 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:57.950984 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:57.950985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:57.950986 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:57.951045 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.951046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.951905 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.951906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.953273 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:57.953277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.954187 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.954189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:57.954291 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.954298 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 04:02:57.954368 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:02:57.954417 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":160,"thread_start_us":107,"threads_started":1}
I20260926 04:02:57.955049 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4ec3b7827645478e63e78f02d1b24e" candidate_uuid: "bb2ee886a2f54312bd82d53938525fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" is_pre_election: true
I20260926 04:02:57.955242 2214 raft_consensus.cc:2454] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2ee886a2f54312bd82d53938525fa7 in term 0.
I20260926 04:02:57.955333 2354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4ec3b7827645478e63e78f02d1b24e" candidate_uuid: "bb2ee886a2f54312bd82d53938525fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce683b3b7f734d61a93b2e4155657fa4" is_pre_election: true
I20260926 04:02:57.955502 2354 raft_consensus.cc:2454] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2ee886a2f54312bd82d53938525fa7 in term 0.
I20260926 04:02:57.955528 2006 leader_election.cc:302] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb2ee886a2f54312bd82d53938525fa7, e84fe4e7683b46fbb66d0e38d2b5457f; no voters:
I20260926 04:02:57.955677 2431 raft_consensus.cc:2781] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:02:57.955752 2431 raft_consensus.cc:482] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:02:57.955772 2431 raft_consensus.cc:3037] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:57.956797 2431 raft_consensus.cc:504] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.957037 2431 leader_election.cc:288] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [CANDIDATE]: Term 1 election: Requested vote from peers e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075), ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131:37409)
I20260926 04:02:57.957126 2354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4ec3b7827645478e63e78f02d1b24e" candidate_uuid: "bb2ee886a2f54312bd82d53938525fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce683b3b7f734d61a93b2e4155657fa4"
I20260926 04:02:57.957132 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc4ec3b7827645478e63e78f02d1b24e" candidate_uuid: "bb2ee886a2f54312bd82d53938525fa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
I20260926 04:02:57.957206 2354 raft_consensus.cc:3037] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:57.957208 2214 raft_consensus.cc:3037] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:57.958108 2214 raft_consensus.cc:2454] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2ee886a2f54312bd82d53938525fa7 in term 1.
I20260926 04:02:57.958117 2354 raft_consensus.cc:2454] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2ee886a2f54312bd82d53938525fa7 in term 1.
I20260926 04:02:57.958312 2007 leader_election.cc:302] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb2ee886a2f54312bd82d53938525fa7, ce683b3b7f734d61a93b2e4155657fa4; no voters:
I20260926 04:02:57.958461 2431 raft_consensus.cc:2781] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:57.958772 2431 raft_consensus.cc:686] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 1 LEADER]: Becoming Leader. State: Replica: bb2ee886a2f54312bd82d53938525fa7, State: Running, Role: LEADER
I20260926 04:02:57.958877 2431 consensus_queue.cc:237] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } }
I20260926 04:02:57.959805 1917 catalog_manager.cc:5495] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129). New cstate: current_term: 1 leader_uuid: "bb2ee886a2f54312bd82d53938525fa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } health_report { overall_health: HEALTHY } } }
I20260926 04:02:57.978583 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:57.976012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.0.200.129:43279)
0926 04:02:57.976090 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:57.976092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:57.976104 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:02:57.976155 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:57.976156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:57.976171 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:57.976243 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:02:57.976363 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.976840 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.977657 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:57.977855 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:57.977947 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:57.978172 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:02:57.978285 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:02:57.978379 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:57.978426 (+ 47us) server_negotiation.cc:299] Negotiation successful
0926 04:02:57.978451 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 04:02:57.980516 2214 raft_consensus.cc:1261] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Refusing update from remote peer bb2ee886a2f54312bd82d53938525fa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:57.980528 2354 raft_consensus.cc:1261] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Refusing update from remote peer bb2ee886a2f54312bd82d53938525fa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:57.980908 2433 consensus_queue.cc:1035] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:57.981047 2431 consensus_queue.cc:1035] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:57.983696 2438 mvcc.cc:205] Tried to move back new op lower bound from 7333459468205191168 to 7333459468120059904. Current Snapshot: MvccSnapshot[applied={T|T < 7333459468205191168}]
I20260926 04:02:57.988066 1917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:02:57.988196 1917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:02:57.989027 1917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Sending DeleteTablet for 3 replicas of tablet fc4ec3b7827645478e63e78f02d1b24e
I20260926 04:02:57.989390 2194 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4ec3b7827645478e63e78f02d1b24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:57 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45454
I20260926 04:02:57.989478 2054 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4ec3b7827645478e63e78f02d1b24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:57 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45726
I20260926 04:02:57.989492 2334 tablet_service.cc:1514] Processing DeleteTablet for tablet fc4ec3b7827645478e63e78f02d1b24e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:57 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:52284
I20260926 04:02:57.989840 2447 tablet_replica.cc:330] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: stopping tablet replica
I20260926 04:02:57.989970 2447 raft_consensus.cc:2229] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:02:57.990069 2449 tablet_replica.cc:330] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: stopping tablet replica
I20260926 04:02:57.990098 2447 raft_consensus.cc:2258] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:57.990163 2449 raft_consensus.cc:2229] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:02:57.990128 1917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:02:57.990270 1917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:02:57.990281 2449 raft_consensus.cc:2258] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:57.990613 2447 ts_tablet_manager.cc:1905] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:57.991636 2194 tablet_service.cc:1467] Processing CreateTablet for tablet 37c08767301a4d548644b7006bce47d1 (DEFAULT_TABLE table=test-table [id=86fead02ac844b5d86771ecb3890ca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.991694 2054 tablet_service.cc:1467] Processing CreateTablet for tablet 37c08767301a4d548644b7006bce47d1 (DEFAULT_TABLE table=test-table [id=86fead02ac844b5d86771ecb3890ca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.991761 2194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37c08767301a4d548644b7006bce47d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.991789 2054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37c08767301a4d548644b7006bce47d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.992439 2447 ts_tablet_manager.cc:1918] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:02:57.992508 2447 log.cc:1196] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/fc4ec3b7827645478e63e78f02d1b24e
I20260926 04:02:57.993090 2428 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Bootstrap starting.
I20260926 04:02:57.993103 2424 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Bootstrap starting.
I20260926 04:02:57.993250 2447 ts_tablet_manager.cc:1939] T fc4ec3b7827645478e63e78f02d1b24e P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting consensus metadata
I20260926 04:02:57.993525 2424 tablet_bootstrap.cc:654] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.993526 2428 tablet_bootstrap.cc:654] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.993988 1902 catalog_manager.cc:4841] TS e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075): tablet fc4ec3b7827645478e63e78f02d1b24e (table test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]) successfully deleted
I20260926 04:02:57.994060 2334 tablet_service.cc:1467] Processing CreateTablet for tablet 37c08767301a4d548644b7006bce47d1 (DEFAULT_TABLE table=test-table [id=86fead02ac844b5d86771ecb3890ca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:02:57.994187 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37c08767301a4d548644b7006bce47d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:57.994186 2428 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: No bootstrap required, opened a new log
I20260926 04:02:57.994233 2428 ts_tablet_manager.cc:1397] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:57.994378 2428 raft_consensus.cc:348] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.994454 2428 raft_consensus.cc:374] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.994477 2428 raft_consensus.cc:729] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2ee886a2f54312bd82d53938525fa7, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.994518 2428 consensus_queue.cc:260] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.994642 2428 ts_tablet_manager.cc:1428] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:02:57.994843 2449 ts_tablet_manager.cc:1905] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:57.995158 2424 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: No bootstrap required, opened a new log
I20260926 04:02:57.995213 2424 ts_tablet_manager.cc:1397] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:02:57.995358 2424 raft_consensus.cc:348] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.995431 2424 raft_consensus.cc:374] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.995457 2424 raft_consensus.cc:729] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84fe4e7683b46fbb66d0e38d2b5457f, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.995497 2424 consensus_queue.cc:260] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.995590 2424 ts_tablet_manager.cc:1428] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:57.995777 2423 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Bootstrap starting.
I20260926 04:02:57.996551 2423 tablet_bootstrap.cc:654] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:57.996958 2448 tablet_replica.cc:330] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: stopping tablet replica
I20260926 04:02:57.996999 2449 ts_tablet_manager.cc:1918] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:02:57.997083 2448 raft_consensus.cc:2229] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:02:57.997174 2449 log.cc:1196] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/fc4ec3b7827645478e63e78f02d1b24e
I20260926 04:02:57.997213 2448 raft_consensus.cc:2258] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:57.997556 2449 ts_tablet_manager.cc:1939] T fc4ec3b7827645478e63e78f02d1b24e P bb2ee886a2f54312bd82d53938525fa7: Deleting consensus metadata
I20260926 04:02:57.997754 2448 ts_tablet_manager.cc:1905] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:57.998148 1903 catalog_manager.cc:4841] TS bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279): tablet fc4ec3b7827645478e63e78f02d1b24e (table test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]) successfully deleted
I20260926 04:02:57.998512 2423 tablet_bootstrap.cc:492] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: No bootstrap required, opened a new log
I20260926 04:02:57.998631 2423 ts_tablet_manager.cc:1397] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 04:02:57.998809 2423 raft_consensus.cc:348] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.999017 2423 raft_consensus.cc:374] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:57.999068 2423 raft_consensus.cc:729] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce683b3b7f734d61a93b2e4155657fa4, State: Initialized, Role: FOLLOWER
I20260926 04:02:57.999112 2423 consensus_queue.cc:260] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:57.999217 2423 ts_tablet_manager.cc:1428] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:02:57.999464 2448 ts_tablet_manager.cc:1918] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:02:57.999534 2448 log.cc:1196] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/fc4ec3b7827645478e63e78f02d1b24e
I20260926 04:02:57.999814 2448 ts_tablet_manager.cc:1939] T fc4ec3b7827645478e63e78f02d1b24e P ce683b3b7f734d61a93b2e4155657fa4: Deleting consensus metadata
I20260926 04:02:58.000227 1903 catalog_manager.cc:4841] TS ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131:37409): tablet fc4ec3b7827645478e63e78f02d1b24e (table test-table [id=3ff3a6a4756b40578ed3deefa188b5bc]) successfully deleted
W20260926 04:02:58.016911 2121 tablet.cc:2369] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:02:58.101190 2401 tablet.cc:2369] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:02:58.316879 2430 raft_consensus.cc:482] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:02:58.316983 2430 raft_consensus.cc:504] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.317373 2430 leader_election.cc:288] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279), e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075)
I20260926 04:02:58.320616 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.317626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35075 (local address 127.0.200.131:37507)
0926 04:02:58.317790 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:58.317799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:58.317837 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:58.317971 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:58.317972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:58.317973 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:58.318097 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.318099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.318840 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.318841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.319698 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.319700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.320308 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.320308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.320386 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.320392 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:02:58.320448 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 04:02:58.320481 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":28,"thread_start_us":61,"threads_started":1}
I20260926 04:02:58.320588 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.317771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:37507 (local address 127.0.200.130:35075)
0926 04:02:58.317839 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.317841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.317869 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:02:58.317894 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.317895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.317903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.317960 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:58.318186 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.318761 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.319778 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.320225 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.320287 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.320294 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.320465 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.320504 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:02:58.320699 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.317663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:58815 (local address 127.0.200.129:43279)
0926 04:02:58.317732 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.317734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.317742 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:02:58.317771 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.317772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.317783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.317834 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:02:58.318047 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.318726 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.319822 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.320299 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.320370 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.320379 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.320585 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.320627 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 04:02:58.320935 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.317569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43279 (local address 127.0.200.131:58815)
0926 04:02:58.317629 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:58.317642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:58.317691 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:58.317877 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:58.317879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:58.317882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:02:58.317963 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.317967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.318839 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.318840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.319719 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.319721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.320382 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.320383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.320491 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.320495 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:02:58.320554 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 04:02:58.320597 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":34}
I20260926 04:02:58.320926 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37c08767301a4d548644b7006bce47d1" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" is_pre_election: true
I20260926 04:02:58.321022 2214 raft_consensus.cc:2454] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 0.
I20260926 04:02:58.321206 2074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37c08767301a4d548644b7006bce47d1" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2ee886a2f54312bd82d53938525fa7" is_pre_election: true
I20260926 04:02:58.321255 2286 leader_election.cc:302] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce683b3b7f734d61a93b2e4155657fa4, e84fe4e7683b46fbb66d0e38d2b5457f; no voters:
I20260926 04:02:58.321352 2074 raft_consensus.cc:2454] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 0.
I20260926 04:02:58.321395 2430 raft_consensus.cc:2781] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:02:58.321437 2430 raft_consensus.cc:482] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:02:58.321457 2430 raft_consensus.cc:3037] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.322371 2430 raft_consensus.cc:504] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.322515 2430 leader_election.cc:288] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 election: Requested vote from peers bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279), e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075)
I20260926 04:02:58.322719 2074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37c08767301a4d548644b7006bce47d1" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2ee886a2f54312bd82d53938525fa7"
I20260926 04:02:58.322748 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37c08767301a4d548644b7006bce47d1" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
I20260926 04:02:58.322813 2214 raft_consensus.cc:3037] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.322805 2074 raft_consensus.cc:3037] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.323622 2214 raft_consensus.cc:2454] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 1.
I20260926 04:02:58.323697 2074 raft_consensus.cc:2454] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 1.
I20260926 04:02:58.323802 2286 leader_election.cc:302] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce683b3b7f734d61a93b2e4155657fa4, e84fe4e7683b46fbb66d0e38d2b5457f; no voters:
I20260926 04:02:58.323895 2430 raft_consensus.cc:2781] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:58.324150 2430 raft_consensus.cc:686] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 LEADER]: Becoming Leader. State: Replica: ce683b3b7f734d61a93b2e4155657fa4, State: Running, Role: LEADER
I20260926 04:02:58.324237 2430 consensus_queue.cc:237] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.325076 1917 catalog_manager.cc:5495] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } health_report { overall_health: HEALTHY } } }
Sep 26 04:02:58 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:02:58.379294 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.371545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.0.200.190:40817)
0926 04:02:58.371664 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.371667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.371795 (+ 128us) server_negotiation.cc:407] Connection header received
0926 04:02:58.371927 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.371927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.371937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.372011 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:58.372503 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.373057 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.373803 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.373992 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.374082 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.376634 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:58.376635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:58.376636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:58.376642 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:58.378000 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.378423 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.378424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.378425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.378479 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.378717 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.378718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.378719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.378836 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:58.378915 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.379094 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.379179 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 04:02:58.379385 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.371457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40817 (local address 127.0.0.1:42058)
0926 04:02:58.371698 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:58.371713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:58.371816 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:58.372029 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:58.372030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:58.372356 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:02:58.372404 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.372406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.373154 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.373155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.373707 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.373710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.374084 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.374084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.374135 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.374840 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:02:58.374849 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:02:58.376491 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:02:58.378107 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:58.378116 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:58.378119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:58.378327 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:58.378577 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:02:58.378578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:02:58.378578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:02:58.378642 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:02:58.378904 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:02:58.378912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:02:58.378958 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:02:58.379068 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 04:02:58.379259 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":85,"thread_start_us":112,"threads_started":1}
I20260926 04:02:58.381665 1917 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:42058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 04:02:58.381875 1917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:02:58.383482 2334 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=41c065f77bdb4520a277ff70d7f9482b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:02:58.383486 2054 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=41c065f77bdb4520a277ff70d7f9482b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:02:58.383674 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:58.383689 2054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:58.385046 2428 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: Bootstrap starting.
I20260926 04:02:58.385174 2194 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=41c065f77bdb4520a277ff70d7f9482b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:02:58.385326 2194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b0c72bcb04d48cea665d2c9e8c7bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:58.385417 2428 tablet_bootstrap.cc:654] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:58.385990 2428 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: No bootstrap required, opened a new log
I20260926 04:02:58.386034 2428 ts_tablet_manager.cc:1397] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:58.386153 2428 raft_consensus.cc:348] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.386227 2428 raft_consensus.cc:374] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:58.386243 2428 raft_consensus.cc:729] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2ee886a2f54312bd82d53938525fa7, State: Initialized, Role: FOLLOWER
I20260926 04:02:58.386303 2428 consensus_queue.cc:260] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.386399 2428 ts_tablet_manager.cc:1428] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:58.386658 2445 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.386747 2445 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.386853 2423 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: Bootstrap starting.
I20260926 04:02:58.387090 2424 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: Bootstrap starting.
I20260926 04:02:58.387252 2423 tablet_bootstrap.cc:654] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:58.387522 2424 tablet_bootstrap.cc:654] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:58.387938 2423 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: No bootstrap required, opened a new log
I20260926 04:02:58.387996 2423 ts_tablet_manager.cc:1397] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:02:58.388155 2424 tablet_bootstrap.cc:492] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: No bootstrap required, opened a new log
I20260926 04:02:58.388146 2423 raft_consensus.cc:348] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.388211 2424 ts_tablet_manager.cc:1397] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:58.388224 2423 raft_consensus.cc:374] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:58.388249 2423 raft_consensus.cc:729] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce683b3b7f734d61a93b2e4155657fa4, State: Initialized, Role: FOLLOWER
I20260926 04:02:58.388307 2423 consensus_queue.cc:260] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.388401 2423 ts_tablet_manager.cc:1428] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 04:02:58.388366 2424 raft_consensus.cc:348] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.388430 2424 raft_consensus.cc:374] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:02:58.388463 2424 raft_consensus.cc:729] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84fe4e7683b46fbb66d0e38d2b5457f, State: Initialized, Role: FOLLOWER
I20260926 04:02:58.388515 2424 consensus_queue.cc:260] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.388530 2430 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.388604 2424 ts_tablet_manager.cc:1428] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:58.388602 2430 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.388659 2427 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.388756 2427 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.538064 2430 raft_consensus.cc:482] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:02:58.538200 2430 raft_consensus.cc:504] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.538435 2430 leader_election.cc:288] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279), e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075)
I20260926 04:02:58.538757 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c72bcb04d48cea665d2c9e8c7bb93" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" is_pre_election: true
I20260926 04:02:58.538756 2074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c72bcb04d48cea665d2c9e8c7bb93" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2ee886a2f54312bd82d53938525fa7" is_pre_election: true
I20260926 04:02:58.538866 2214 raft_consensus.cc:2454] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 0.
I20260926 04:02:58.538866 2074 raft_consensus.cc:2454] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 0.
I20260926 04:02:58.539036 2286 leader_election.cc:302] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce683b3b7f734d61a93b2e4155657fa4, e84fe4e7683b46fbb66d0e38d2b5457f; no voters:
I20260926 04:02:58.539199 2430 raft_consensus.cc:2781] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:02:58.539247 2430 raft_consensus.cc:482] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:02:58.539263 2430 raft_consensus.cc:3037] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.539925 2430 raft_consensus.cc:504] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.540077 2430 leader_election.cc:288] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 election: Requested vote from peers bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279), e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075)
I20260926 04:02:58.540274 2214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c72bcb04d48cea665d2c9e8c7bb93" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f"
I20260926 04:02:58.540364 2214 raft_consensus.cc:3037] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.540290 2074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0c72bcb04d48cea665d2c9e8c7bb93" candidate_uuid: "ce683b3b7f734d61a93b2e4155657fa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2ee886a2f54312bd82d53938525fa7"
I20260926 04:02:58.540412 2074 raft_consensus.cc:3037] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:02:58.540999 2214 raft_consensus.cc:2454] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 1.
I20260926 04:02:58.541260 2074 raft_consensus.cc:2454] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce683b3b7f734d61a93b2e4155657fa4 in term 1.
I20260926 04:02:58.541282 2286 leader_election.cc:302] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce683b3b7f734d61a93b2e4155657fa4, e84fe4e7683b46fbb66d0e38d2b5457f; no voters:
I20260926 04:02:58.541409 2430 raft_consensus.cc:2781] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:02:58.541541 2430 raft_consensus.cc:686] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 LEADER]: Becoming Leader. State: Replica: ce683b3b7f734d61a93b2e4155657fa4, State: Running, Role: LEADER
I20260926 04:02:58.541612 2430 consensus_queue.cc:237] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } }
I20260926 04:02:58.541926 2454 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: TxnStatusTablet state changed. Reason: New leader ce683b3b7f734d61a93b2e4155657fa4. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.542034 2454 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:02:58.542246 2466 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 04:02:58.542418 2466 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 04:02:58.542400 1917 catalog_manager.cc:5495] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } health_report { overall_health: HEALTHY } } }
I20260926 04:02:58.573948 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.570425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.0.200.131:37409)
0926 04:02:58.570664 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.570666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.570790 (+ 124us) server_negotiation.cc:407] Connection header received
0926 04:02:58.570938 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.570939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.570966 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.571070 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:02:58.571244 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.571974 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.572900 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.573192 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.573325 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.573563 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:02:58.573676 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:02:58.573770 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.573819 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.573843 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":70,"threads_started":1}
I20260926 04:02:58.573936 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.570616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:37409 (local address 127.0.0.1:52300)
0926 04:02:58.570687 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:58.570700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:58.570780 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:58.571089 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:58.571091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:58.571092 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:02:58.571150 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.571154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.572097 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.572099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.572775 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.572777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.573335 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.573336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.573406 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.573436 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:02:58.573758 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:02:58.573768 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 04:02:58.573805 (+ 37us) client_negotiation.cc:240] Negotiation successful
0926 04:02:58.573865 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 04:02:58.575587 2214 raft_consensus.cc:1261] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Refusing update from remote peer ce683b3b7f734d61a93b2e4155657fa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:58.575693 2074 raft_consensus.cc:1261] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Refusing update from remote peer ce683b3b7f734d61a93b2e4155657fa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:58.575878 2430 consensus_queue.cc:1035] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:58.576030 2454 consensus_queue.cc:1035] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:58.576648 2427 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: TxnStatusTablet state changed. Reason: New leader ce683b3b7f734d61a93b2e4155657fa4. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.576748 2427 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.577111 2445 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: TxnStatusTablet state changed. Reason: New leader ce683b3b7f734d61a93b2e4155657fa4. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.577224 2445 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.577462 2454 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.577561 2454 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:02:58.577629 2470 mvcc.cc:205] Tried to move back new op lower bound from 7333459470642995200 to 7333459470506897408. Current Snapshot: MvccSnapshot[applied={T|T < 7333459470642995200}]
I20260926 04:02:58.578429 2430 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.578524 2430 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P ce683b3b7f734d61a93b2e4155657fa4: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:02:58.579061 2427 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.579057 2445 tablet_replica.cc:439] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce683b3b7f734d61a93b2e4155657fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 } } peers { permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 } } peers { permanent_uuid: "ce683b3b7f734d61a93b2e4155657fa4" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 37409 } } }
I20260926 04:02:58.579149 2427 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P e84fe4e7683b46fbb66d0e38d2b5457f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.579150 2445 tablet_replica.cc:442] T 2b0c72bcb04d48cea665d2c9e8c7bb93 P bb2ee886a2f54312bd82d53938525fa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:02:58.579622 2473 mvcc.cc:205] Tried to move back new op lower bound from 7333459470642995200 to 7333459470506897408. Current Snapshot: MvccSnapshot[applied={T|T < 7333459470642995200}]
I20260926 04:02:58.583458 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.580446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.0.200.131:37409)
0926 04:02:58.580487 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.580489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.580561 (+ 72us) server_negotiation.cc:407] Connection header received
0926 04:02:58.580857 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.580859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.580874 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.580988 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:02:58.581162 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.581660 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.582534 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.582800 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.582887 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.583186 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:02:58.583216 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:02:58.583282 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.583344 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.583369 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:02:58.590103 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.587506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:57205 (local address 127.0.200.131:37409)
0926 04:02:58.587599 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.587602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.587720 (+ 118us) server_negotiation.cc:407] Connection header received
0926 04:02:58.587787 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.587788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.587808 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.587860 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:02:58.588003 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.588472 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.589298 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.589498 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.589601 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.589797 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:02:58.589843 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:02:58.589910 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.589969 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.589992 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 04:02:58.590121 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.587370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:37409 (local address 127.0.200.131:57205)
0926 04:02:58.587636 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 04:02:58.587647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:02:58.587738 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:02:58.587860 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:02:58.587860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:02:58.587862 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:02:58.587931 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.587933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.588571 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.588572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.589190 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:02:58.589192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.589603 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.589604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:02:58.589669 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.589694 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:02:58.589910 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:02:58.589912 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:02:58.589966 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 04:02:58.590007 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":74,"threads_started":1}
I20260926 04:02:58.592769 2214 raft_consensus.cc:1261] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Refusing update from remote peer ce683b3b7f734d61a93b2e4155657fa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:58.592744 2074 raft_consensus.cc:1261] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Refusing update from remote peer ce683b3b7f734d61a93b2e4155657fa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:02:58.593001 2483 consensus_queue.cc:1035] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2ee886a2f54312bd82d53938525fa7" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:58.593180 2454 consensus_queue.cc:1035] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84fe4e7683b46fbb66d0e38d2b5457f" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:02:58.608958 1916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:02:58.609117 1916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:02:58.609920 1916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e04755f4c14579a23f878915ed8e4f: Sending DeleteTablet for 3 replicas of tablet 37c08767301a4d548644b7006bce47d1
I20260926 04:02:58.610378 2194 tablet_service.cc:1514] Processing DeleteTablet for tablet 37c08767301a4d548644b7006bce47d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:58 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45454
I20260926 04:02:58.610463 2054 tablet_service.cc:1514] Processing DeleteTablet for tablet 37c08767301a4d548644b7006bce47d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:58 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45726
I20260926 04:02:58.610457 2334 tablet_service.cc:1514] Processing DeleteTablet for tablet 37c08767301a4d548644b7006bce47d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:02:58 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:52284
I20260926 04:02:58.610679 2488 tablet_replica.cc:330] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: stopping tablet replica
I20260926 04:02:58.610709 2489 tablet_replica.cc:330] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: stopping tablet replica
I20260926 04:02:58.610728 2488 raft_consensus.cc:2229] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:02:58.610765 2488 raft_consensus.cc:2258] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:58.610761 2489 raft_consensus.cc:2229] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:02:58.610864 2489 raft_consensus.cc:2258] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:58.611050 2488 ts_tablet_manager.cc:1905] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:58.611129 2489 ts_tablet_manager.cc:1905] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:58.612855 2488 ts_tablet_manager.cc:1918] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:02:58.612963 2489 ts_tablet_manager.cc:1918] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:02:58.613044 2489 log.cc:1196] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/37c08767301a4d548644b7006bce47d1
I20260926 04:02:58.613148 2488 log.cc:1196] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/37c08767301a4d548644b7006bce47d1
I20260926 04:02:58.613350 2489 ts_tablet_manager.cc:1939] T 37c08767301a4d548644b7006bce47d1 P ce683b3b7f734d61a93b2e4155657fa4: Deleting consensus metadata
I20260926 04:02:58.613540 2488 ts_tablet_manager.cc:1939] T 37c08767301a4d548644b7006bce47d1 P bb2ee886a2f54312bd82d53938525fa7: Deleting consensus metadata
I20260926 04:02:58.613729 1903 catalog_manager.cc:4841] TS ce683b3b7f734d61a93b2e4155657fa4 (127.0.200.131:37409): tablet 37c08767301a4d548644b7006bce47d1 (table test-table [id=86fead02ac844b5d86771ecb3890ca5a]) successfully deleted
I20260926 04:02:58.613971 1903 catalog_manager.cc:4841] TS bb2ee886a2f54312bd82d53938525fa7 (127.0.200.129:43279): tablet 37c08767301a4d548644b7006bce47d1 (table test-table [id=86fead02ac844b5d86771ecb3890ca5a]) successfully deleted
I20260926 04:02:58.614689 2487 tablet_replica.cc:330] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: stopping tablet replica
I20260926 04:02:58.614768 2487 raft_consensus.cc:2229] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:02:58.614813 2487 raft_consensus.cc:2258] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:02:58.615077 2487 ts_tablet_manager.cc:1905] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:02:58.616801 2487 ts_tablet_manager.cc:1918] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:02:58.616871 2487 log.cc:1196] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/37c08767301a4d548644b7006bce47d1
I20260926 04:02:58.617097 2487 ts_tablet_manager.cc:1939] T 37c08767301a4d548644b7006bce47d1 P e84fe4e7683b46fbb66d0e38d2b5457f: Deleting consensus metadata
I20260926 04:02:58.617657 1902 catalog_manager.cc:4841] TS e84fe4e7683b46fbb66d0e38d2b5457f (127.0.200.130:35075): tablet 37c08767301a4d548644b7006bce47d1 (table test-table [id=86fead02ac844b5d86771ecb3890ca5a]) successfully deleted
Sep 26 04:02:58 dist-test-slave-j8c1 krb5kdc[1872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:02:58.623421 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.612874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.0.200.129:43279)
0926 04:02:58.612928 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.612930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.613103 (+ 173us) server_negotiation.cc:407] Connection header received
0926 04:02:58.613205 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.613207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.613217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.613247 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:58.613947 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.614766 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.615702 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.615914 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.616013 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.619549 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:58.619560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:58.619561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:58.619569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:58.621758 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.622178 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.622182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.622182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.622254 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.622719 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.622721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.622721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.622921 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:58.623131 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.623219 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.623310 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260926 04:02:58.623716 2114 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45750
I20260926 04:02:58.632211 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:58.625158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.0.200.190:40817)
0926 04:02:58.625213 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:58.625215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:58.625225 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:02:58.625276 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:58.625277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:58.625288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:58.625351 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:58.625949 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.626497 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.627547 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:02:58.627788 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:02:58.627883 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:02:58.629320 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:02:58.629322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:02:58.629322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:02:58.629330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:02:58.630881 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.631304 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.631305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.631305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.631366 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:02:58.631606 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:02:58.631607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:02:58.631608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:02:58.631733 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:02:58.631847 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 04:02:58.631953 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 04:02:58.632035 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260926 04:02:58.632516 1916 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42060
I20260926 04:02:58.633500 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1989
I20260926 04:02:58.640348 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2129
I20260926 04:02:58.646534 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2269
I20260926 04:02:58.652269 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 1885
2026-09-26T04:02:58Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2641 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:02:58 dist-test-slave-j8c1 krb5kdc[2496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:02:58 dist-test-slave-j8c1 krb5kdc[2496](info): set up 2 sockets
Sep 26 04:02:58 dist-test-slave-j8c1 krb5kdc[2496](info): commencing operation
krb5kdc: starting...
W20260926 04:02:59.718367 802 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:02:59 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:02:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:02:59.850508 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45621
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:02:59.936842 2509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:02:59.937096 2509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:02:59.937121 2509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:02:59.938480 2509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:02:59.938575 2509 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 04:02:59.938591 2509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:02:59.938604 2509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:02:59.938676 2509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:02:59.938694 2509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:02:59.940388 2509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:45621
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:02:59.940704 2509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:02:59.941032 2509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:02:59.946305 2515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:59.946303 2517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:02:59.946336 2514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:02:59.946810 2509 server_base.cc:1034] running on GCE node
I20260926 04:02:59.947113 2509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:02:59.947412 2509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:02:59.948626 2509 hybrid_clock.cc:648] HybridClock initialized: now 1790395379948609 us; error 30 us; skew 500 ppm
Sep 26 04:02:59 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:02:59.951596 2509 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:02:59.952149 2509 webserver.cc:460] Webserver started at http://127.0.200.190:39619/ using document root <none> and password file <none>
I20260926 04:02:59.952404 2509 fs_manager.cc:359] Metadata directory not provided
I20260926 04:02:59.952481 2509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:02:59.952629 2509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:02:59.953801 2509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "f06fd1ea364c4a68850b5783049f734a"
format_stamp: "Formatted at 2026-09-26 04:02:59 on dist-test-slave-j8c1"
server_key: "0bb1d6d4bada565359ef573096c1027b"
server_key_iv: "f58243d404b314da6d835b775561be53"
server_key_version: "encryptionkey@0"
I20260926 04:02:59.954239 2509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "f06fd1ea364c4a68850b5783049f734a"
format_stamp: "Formatted at 2026-09-26 04:02:59 on dist-test-slave-j8c1"
server_key: "0bb1d6d4bada565359ef573096c1027b"
server_key_iv: "f58243d404b314da6d835b775561be53"
server_key_version: "encryptionkey@0"
I20260926 04:02:59.955971 2509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:02:59.957005 2524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:02:59.957383 2509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:02:59.957516 2509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "f06fd1ea364c4a68850b5783049f734a"
format_stamp: "Formatted at 2026-09-26 04:02:59 on dist-test-slave-j8c1"
server_key: "0bb1d6d4bada565359ef573096c1027b"
server_key_iv: "f58243d404b314da6d835b775561be53"
server_key_version: "encryptionkey@0"
I20260926 04:02:59.957614 2509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:02:59.985790 2509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:02:59.988890 2509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:02:59.989107 2509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:02:59.994339 2509 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:45621
I20260926 04:02:59.994445 2586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:45621 every 8 connection(s)
I20260926 04:02:59.994840 2509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:02:59.995610 2587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:02:59.996677 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2509
I20260926 04:02:59.996765 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:02:59.996985 802 external_mini_cluster.cc:1428] Setting key 219bfcfe90f07c7973c57d1abceb2851
I20260926 04:02:59.998430 2587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Bootstrap starting.
I20260926 04:02:59.999287 2587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Neither blocks nor log segments found. Creating new log.
I20260926 04:02:59.999534 2587 log.cc:824] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:00.000313 2587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: No bootstrap required, opened a new log
I20260926 04:03:00.002647 2587 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } }
I20260926 04:03:00.002856 2587 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.002887 2587 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06fd1ea364c4a68850b5783049f734a, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.003031 2587 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } }
I20260926 04:03:00.003113 2587 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:00.003145 2587 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:00.003232 2587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.004079 2587 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } }
I20260926 04:03:00.004204 2587 leader_election.cc:302] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06fd1ea364c4a68850b5783049f734a; no voters:
I20260926 04:03:00.004448 2587 leader_election.cc:288] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:00.004469 2592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:00.004613 2592 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [term 1 LEADER]: Becoming Leader. State: Replica: f06fd1ea364c4a68850b5783049f734a, State: Running, Role: LEADER
I20260926 04:03:00.004694 2592 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } }
I20260926 04:03:00.004763 2587 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 04:02:59 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.005025 2592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06fd1ea364c4a68850b5783049f734a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } } }
I20260926 04:03:00.005092 2592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:00.005211 2592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06fd1ea364c4a68850b5783049f734a. Latest consensus state: current_term: 1 leader_uuid: "f06fd1ea364c4a68850b5783049f734a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fd1ea364c4a68850b5783049f734a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45621 } } }
I20260926 04:03:00.005277 2592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:00.005792 2596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:00.006341 2596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:00.009461 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:02:59.997573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.0.200.190:45621)
0926 04:02:59.997767 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:02:59.997776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:02:59.997802 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:02:59.998225 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:02:59.998231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:02:59.998282 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:02:59.998406 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:02:59.999285 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.000724 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.001924 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.002355 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.002451 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.005651 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.005655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.005656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.005670 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.007781 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.008202 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.008205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.008206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.008312 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.008550 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.008551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.008551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.008810 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.008968 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.009097 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.009245 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260926 04:03:00.011941 2596 catalog_manager.cc:1347] Generated new cluster ID: 2bbee434199b472eb9b711856bdcb148
I20260926 04:03:00.012005 2596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:00.020884 2596 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:00.021497 2596 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:00.035960 2596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Generated new TSK 0
I20260926 04:03:00.036211 2596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:00.071017 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45621
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:00.162088 2613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:00.162369 2613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:00.162402 2613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:00.163743 2613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:00.163800 2613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:00.163822 2613 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 04:03:00.163834 2613 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:00.163865 2613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:00.165496 2613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:00.165917 2613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:00.166234 2613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:00.172374 2619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.172484 2621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.172327 2618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:00.172540 2613 server_base.cc:1034] running on GCE node
I20260926 04:03:00.172930 2613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:00.173242 2613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:00.174527 2613 hybrid_clock.cc:648] HybridClock initialized: now 1790395380174440 us; error 28 us; skew 500 ppm
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:00.177868 2613 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:00.178438 2613 webserver.cc:460] Webserver started at http://127.0.200.129:38833/ using document root <none> and password file <none>
I20260926 04:03:00.178759 2613 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:00.178838 2613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:00.178997 2613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:00.180106 2613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "1284e3967f7e4f75befda8e10c79f4ae"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "f1f103c1e0293a571092c74c07889464"
server_key_iv: "ab9249c92b3bb045cd54152fe163698e"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.180486 2613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "1284e3967f7e4f75befda8e10c79f4ae"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "f1f103c1e0293a571092c74c07889464"
server_key_iv: "ab9249c92b3bb045cd54152fe163698e"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.181962 2613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:00.182973 2628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.183225 2613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:00.183358 2613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "1284e3967f7e4f75befda8e10c79f4ae"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "f1f103c1e0293a571092c74c07889464"
server_key_iv: "ab9249c92b3bb045cd54152fe163698e"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.183446 2613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:00.200018 2613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:00.202327 2613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:00.202543 2613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:00.203202 2613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:00.203297 2613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.203347 2613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:00.203388 2613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.211052 2613 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:37445
I20260926 04:03:00.211261 2743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:37445 every 8 connection(s)
I20260926 04:03:00.211731 2613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.215711 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.203996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:50079 (local address 127.0.200.190:45621)
0926 04:03:00.204098 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.204101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.204401 (+ 300us) server_negotiation.cc:407] Connection header received
0926 04:03:00.205033 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.205035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.205064 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.205134 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.206121 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.207038 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.209003 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.209498 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.209645 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.212940 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.212942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.212942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.212950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.214225 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.214681 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.214682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.214683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.214757 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.215024 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.215025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.215026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.215158 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.215263 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.215544 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.215636 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:03:00.215888 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.203957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.129:50079)
0926 04:03:00.204262 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.204302 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.204928 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.205258 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.205263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.205675 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.206006 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.206012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.207150 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.207152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.208890 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.208894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.209736 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.209738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.209925 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.210630 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.210646 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.212824 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.214321 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.214327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.214329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.214604 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.214832 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.214833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.214833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.214937 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.215269 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.215273 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.215398 (+ 125us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.215485 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.215618 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":171,"threads_started":1}
I20260926 04:03:00.216454 2744 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45621
I20260926 04:03:00.216548 2744 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:00.216768 2744 heartbeater.cc:511] Master 127.0.200.190:45621 requested a full tablet report, sending...
I20260926 04:03:00.216969 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2613
I20260926 04:03:00.217067 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:00.217264 802 external_mini_cluster.cc:1428] Setting key dbdb29ebca03107d3ab8ed662da2be4e
I20260926 04:03:00.217406 2540 ts_manager.cc:194] Registered new tserver with Master: 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445)
I20260926 04:03:00.218063 2540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:50079
I20260926 04:03:00.223734 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.217747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:56577 (local address 127.0.200.190:45621)
0926 04:03:00.217784 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.217786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.217796 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:00.217823 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.217824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.217833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.217881 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.218213 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.218992 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.219678 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.219882 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.219968 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.220976 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.220978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.220978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.220985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.222284 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.222712 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.222713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.222714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.222777 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.223142 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.223143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.223143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.223271 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.223379 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.223540 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.223634 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:00.223753 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.217542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.129:56577)
0926 04:03:00.217700 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.217712 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.217775 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.217883 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.217884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.218073 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.218123 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.218125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.219074 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.219075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.219579 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.219581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.219969 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.219970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.220044 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.220538 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.220543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.220882 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.222391 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.222392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.222393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.222624 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.222980 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.222981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.222982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.223065 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.223367 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.223368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.223418 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.223515 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.223630 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:00.249794 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45621
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:00.337023 2753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:00.337244 2753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:00.337270 2753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:00.338382 2753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:00.338420 2753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:00.338440 2753 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 04:03:00.338452 2753 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:00.338469 2753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:00.339995 2753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:00.340199 2753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:00.340380 2753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:00.345619 2761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.345602 2758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.345789 2759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:00.346119 2753 server_base.cc:1034] running on GCE node
I20260926 04:03:00.346467 2753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:00.346783 2753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:00.347975 2753 hybrid_clock.cc:648] HybridClock initialized: now 1790395380347964 us; error 35 us; skew 500 ppm
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:00.350809 2753 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:00.351248 2753 webserver.cc:460] Webserver started at http://127.0.200.130:40815/ using document root <none> and password file <none>
I20260926 04:03:00.351451 2753 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:00.351495 2753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:00.351588 2753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:00.352660 2753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "266115e0ea70411e89992561b8f90233"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "2cd8037838777a216587475b3b2398ab"
server_key_iv: "c07d144ce02b578c4e30d89cc0b36076"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.352983 2753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "266115e0ea70411e89992561b8f90233"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "2cd8037838777a216587475b3b2398ab"
server_key_iv: "c07d144ce02b578c4e30d89cc0b36076"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.354346 2753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 04:03:00.355252 2768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.355440 2753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:00.355532 2753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "266115e0ea70411e89992561b8f90233"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "2cd8037838777a216587475b3b2398ab"
server_key_iv: "c07d144ce02b578c4e30d89cc0b36076"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.355595 2753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:00.368345 2753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:00.370818 2753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:00.370944 2753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:00.371429 2753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:00.371487 2753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.371526 2753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:00.371585 2753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.379004 2753 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:43937
I20260926 04:03:00.379127 2883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:43937 every 8 connection(s)
I20260926 04:03:00.379454 2753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:00.381822 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.372019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60003 (local address 127.0.200.190:45621)
0926 04:03:00.372099 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.372101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.372283 (+ 182us) server_negotiation.cc:407] Connection header received
0926 04:03:00.372766 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.372767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.372782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.372875 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.373767 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.374643 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.375861 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.376197 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.376270 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.379088 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.379090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.379090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.379098 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.380607 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.381022 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.381023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.381023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.381073 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.381250 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.381250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.381250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.381353 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.381404 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.381670 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.381732 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 04:03:00.381969 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.371979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.130:60003)
0926 04:03:00.372143 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.372186 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.372664 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.372929 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.372932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.373335 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.373641 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.373650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.374775 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.374776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.375769 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.375771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.376284 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.376285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.376430 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.377078 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.377088 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.378957 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.380683 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.380688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.380690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.380951 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.381114 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.381115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.381115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.381199 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.381411 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.381416 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.381531 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.381615 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.381738 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260926 04:03:00.382519 2884 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45621
I20260926 04:03:00.382623 2884 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:00.382817 2884 heartbeater.cc:511] Master 127.0.200.190:45621 requested a full tablet report, sending...
I20260926 04:03:00.383227 2540 ts_manager.cc:194] Registered new tserver with Master: 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937)
I20260926 04:03:00.383672 2540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:60003
I20260926 04:03:00.385106 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2753
I20260926 04:03:00.385212 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:00.385396 802 external_mini_cluster.cc:1428] Setting key 06f22952125d500b4fad6d711109b281
I20260926 04:03:00.390350 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.383587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.130:43697)
0926 04:03:00.383818 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.383830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.384018 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.384154 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.384155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.384344 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.384396 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.384397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.385075 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.385076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.385842 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.385845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.386301 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.386302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.386362 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.386786 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.386790 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.387133 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.389198 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.389199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.389200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.389374 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.389624 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.389625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.389625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.389690 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.390041 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.390042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.390085 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.390144 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.390234 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":96,"threads_started":1}
I20260926 04:03:00.390439 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.383680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43697 (local address 127.0.200.190:45621)
0926 04:03:00.383730 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.383732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.384000 (+ 268us) server_negotiation.cc:407] Connection header received
0926 04:03:00.384080 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.384081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.384092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.384148 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.384490 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.384978 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.385925 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.386213 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.386288 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.387221 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.387223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.387223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.387230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.389084 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.389461 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.389462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.389462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.389524 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.389774 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.389775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.389775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.389983 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.390051 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.390204 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.390296 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:00.414480 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45621
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:00.499243 2893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:00.499456 2893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:00.499483 2893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:00.500715 2893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:00.500766 2893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:00.500788 2893 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 04:03:00.500800 2893 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:00.500861 2893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:00.502419 2893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:00.502718 2893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:00.503001 2893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:00.508441 2901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.508510 2898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:00.508510 2899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:00.508982 2893 server_base.cc:1034] running on GCE node
I20260926 04:03:00.509194 2893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:00.509452 2893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:00.510628 2893 hybrid_clock.cc:648] HybridClock initialized: now 1790395380510605 us; error 38 us; skew 500 ppm
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:00.513278 2893 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:00.513777 2893 webserver.cc:460] Webserver started at http://127.0.200.131:34923/ using document root <none> and password file <none>
I20260926 04:03:00.514038 2893 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:00.514086 2893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:00.514247 2893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:00.515441 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "8ac6b3d44247404b8d005a5a7afb584b"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "a3f9af28ac9227aae64e0acc57bb09ef"
server_key_iv: "1787e5b3f8cc3e4d602bc8d371c937d8"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.515827 2893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "8ac6b3d44247404b8d005a5a7afb584b"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "a3f9af28ac9227aae64e0acc57bb09ef"
server_key_iv: "1787e5b3f8cc3e4d602bc8d371c937d8"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.517253 2893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:03:00.518160 2908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.518456 2893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:00.518536 2893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "8ac6b3d44247404b8d005a5a7afb584b"
format_stamp: "Formatted at 2026-09-26 04:03:00 on dist-test-slave-j8c1"
server_key: "a3f9af28ac9227aae64e0acc57bb09ef"
server_key_iv: "1787e5b3f8cc3e4d602bc8d371c937d8"
server_key_version: "encryptionkey@0"
I20260926 04:03:00.518669 2893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:00.530893 2893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:00.533008 2893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:00.533174 2893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:00.533727 2893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:00.533771 2893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.533797 2893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:00.533865 2893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.541656 2893 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:44091
I20260926 04:03:00.542156 2893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:00.542178 3023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:44091 every 8 connection(s)
I20260926 04:03:00.544478 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.534326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33705 (local address 127.0.200.190:45621)
0926 04:03:00.534387 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.534390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.534684 (+ 294us) server_negotiation.cc:407] Connection header received
0926 04:03:00.535291 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.535293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.535315 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.535378 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.536260 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.537275 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.538386 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.538705 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.538804 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.541161 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.541163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.541163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.541172 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.542961 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.543426 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.543428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.543429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.543497 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.543780 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.543781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.543781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.543913 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.543998 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.544300 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.544385 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:03:00.544711 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.534300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.131:33705)
0926 04:03:00.534494 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.534530 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.535171 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.535453 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.535458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.535847 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.536141 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.536148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.537414 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.537415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.538273 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.538276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.538825 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.538827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.538972 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.539570 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.539580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.541048 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.543072 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.543078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.543081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.543321 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.543578 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.543579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.543579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.543700 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.543988 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.543994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.544147 (+ 153us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.544264 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.544437 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":126,"threads_started":1}
I20260926 04:03:00.545329 3024 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45621
I20260926 04:03:00.545424 3024 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:00.545620 3024 heartbeater.cc:511] Master 127.0.200.190:45621 requested a full tablet report, sending...
I20260926 04:03:00.546085 2541 ts_manager.cc:194] Registered new tserver with Master: 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:00.546751 2541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:33705
I20260926 04:03:00.549795 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 2893
I20260926 04:03:00.549933 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:00.550114 802 external_mini_cluster.cc:1428] Setting key 89d3850286b80d80cc6420e67d9123c5
I20260926 04:03:00.551151 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:03:00.552752 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.546522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:40309 (local address 127.0.200.190:45621)
0926 04:03:00.546592 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.546594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.546675 (+ 81us) server_negotiation.cc:407] Connection header received
0926 04:03:00.546753 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.546754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.546762 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.546806 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.547186 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.547664 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.548362 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.548537 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.548589 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.549520 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.549521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.549521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.549527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.551567 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.552036 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.552037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.552038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.552107 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.552319 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.552320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.552320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.552421 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.552471 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.552633 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.552689 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 04:03:00.552915 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.546453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.200.131:40309)
0926 04:03:00.546625 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.546635 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.546689 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.546822 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.546823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.547014 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.547105 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.547107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.547768 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.547769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.548286 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.548287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.548604 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.548605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.548662 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.549121 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.549125 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.549439 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.551656 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.551657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.551658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.551956 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.552180 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.552181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.552181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.552255 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.552485 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.552486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.552541 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.552617 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.552740 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.576169 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.567877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44768 (local address 127.0.200.190:45621)
0926 04:03:00.567940 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.567942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.567949 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:03:00.567972 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.567972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.567981 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.568050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.568662 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.569205 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.569987 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.570179 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.570264 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.572904 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.572905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.572905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.572914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.574728 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.575127 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.575128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.575128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.575209 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.575494 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.575498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.575498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.575631 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.575736 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.575946 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.576016 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 04:03:00.578163 2541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:00.579960 2541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:00.587466 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.583016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.0.200.131:44091)
0926 04:03:00.583208 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.583213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.583222 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:00.583268 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.583269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.583282 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.583383 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.583718 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.584479 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.586108 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.586983 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587067 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587132 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.587321 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.587364 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260926 04:03:00.587522 3042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.582764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.0.1:48160)
0926 04:03:00.583085 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.583095 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.583133 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.583393 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.583396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.583398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.583475 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.583477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.584587 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.584588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.586028 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.586030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.587089 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.587178 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587195 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.587280 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.587367 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":70,"threads_started":1}
I20260926 04:03:00.587630 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.582718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43937 (local address 127.0.0.1:55466)
0926 04:03:00.582833 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.582882 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.582957 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.583513 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.583515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.583518 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.583577 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.583580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.585001 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.585003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.585974 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.585976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.587422 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.587481 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587484 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.587515 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.587558 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":18,"thread_start_us":57,"threads_started":1}
I20260926 04:03:00.587989 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.583070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.0.200.129:37445)
0926 04:03:00.583255 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.583260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.583272 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:00.583314 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.583316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.583481 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.583552 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.583732 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.584873 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.586228 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.586983 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587071 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587132 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.587471 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.587518 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260926 04:03:00.588048 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.582831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.0.200.130:43937)
0926 04:03:00.583346 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.583351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.583361 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:00.583395 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.583397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.583412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.583656 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.583715 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.584897 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.586616 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.587312 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587819 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587889 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.587916 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.587958 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":96,"threads_started":1}
I20260926 04:03:00.588400 2958 tablet_service.cc:1467] Processing CreateTablet for tablet d51d7714892945ebbe92aa61321e47df (DEFAULT_TABLE table=test-table [id=e2c0ef15120346fb81b836c2ce87e18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:00.588791 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51d7714892945ebbe92aa61321e47df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.588739 2818 tablet_service.cc:1467] Processing CreateTablet for tablet d51d7714892945ebbe92aa61321e47df (DEFAULT_TABLE table=test-table [id=e2c0ef15120346fb81b836c2ce87e18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:00.589080 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51d7714892945ebbe92aa61321e47df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.590992 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.582963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37445 (local address 127.0.0.1:55076)
0926 04:03:00.583122 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.583134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.583186 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.583615 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.583616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.583618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.583649 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.583650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.584976 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.584977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.586146 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.586148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.587187 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.587188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.587254 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.587258 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.587309 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.587364 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":34,"thread_start_us":63,"threads_started":1}
I20260926 04:03:00.591810 2678 tablet_service.cc:1467] Processing CreateTablet for tablet d51d7714892945ebbe92aa61321e47df (DEFAULT_TABLE table=test-table [id=e2c0ef15120346fb81b836c2ce87e18e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:00.592113 3047 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Bootstrap starting.
I20260926 04:03:00.592167 2678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d51d7714892945ebbe92aa61321e47df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.592618 3048 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Bootstrap starting.
I20260926 04:03:00.593067 3047 tablet_bootstrap.cc:654] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.593276 3048 tablet_bootstrap.cc:654] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.593318 3047 log.cc:824] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:00.593614 3048 log.cc:824] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:00.594348 3048 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: No bootstrap required, opened a new log
I20260926 04:03:00.594398 3048 ts_tablet_manager.cc:1397] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:00.594542 3047 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: No bootstrap required, opened a new log
I20260926 04:03:00.594647 3047 ts_tablet_manager.cc:1397] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:03:00.595892 3048 raft_consensus.cc:348] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.596065 3048 raft_consensus.cc:374] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.596112 3048 raft_consensus.cc:729] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266115e0ea70411e89992561b8f90233, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.596225 3048 consensus_queue.cc:260] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.596536 3048 ts_tablet_manager.cc:1428] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:00.596562 2884 heartbeater.cc:503] Master 127.0.200.190:45621 was elected leader, sending a full tablet report...
I20260926 04:03:00.596809 3047 raft_consensus.cc:348] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.596993 3047 raft_consensus.cc:374] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.597035 3047 raft_consensus.cc:729] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.597142 3047 consensus_queue.cc:260] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.597481 3047 ts_tablet_manager.cc:1428] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:03:00.597520 3024 heartbeater.cc:503] Master 127.0.200.190:45621 was elected leader, sending a full tablet report...
I20260926 04:03:00.597678 3051 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Bootstrap starting.
I20260926 04:03:00.598407 3051 tablet_bootstrap.cc:654] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.598754 3051 log.cc:824] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:00.599522 3051 tablet_bootstrap.cc:492] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: No bootstrap required, opened a new log
I20260926 04:03:00.599638 3051 ts_tablet_manager.cc:1397] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:00.601049 3051 raft_consensus.cc:348] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.601225 3051 raft_consensus.cc:374] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.601274 3051 raft_consensus.cc:729] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.601397 3051 consensus_queue.cc:260] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.601716 3051 ts_tablet_manager.cc:1428] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:00.601852 2744 heartbeater.cc:503] Master 127.0.200.190:45621 was elected leader, sending a full tablet report...
I20260926 04:03:00.614892 3055 raft_consensus.cc:482] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:00.615061 3055 raft_consensus.cc:504] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.615494 3055 leader_election.cc:288] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:00.618692 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.615801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:47567 (local address 127.0.200.131:44091)
0926 04:03:00.615844 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.615846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.615857 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:00.615889 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.615890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.615897 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.615947 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.616168 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.616780 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.617917 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.618309 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.618367 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.618374 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.618570 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.618614 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:03:00.618731 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.615702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.200.129:47567)
0926 04:03:00.615743 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.615756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.615801 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.615978 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.615980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.615981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.616071 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.616073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.616863 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.616864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.617813 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.617815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.618396 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.618397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.618483 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.618487 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.618539 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.618627 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":39}
I20260926 04:03:00.618767 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.615763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55469 (local address 127.0.200.130:43937)
0926 04:03:00.615806 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.615808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.616154 (+ 346us) server_negotiation.cc:407] Connection header received
0926 04:03:00.616196 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.616197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.616206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.616259 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.616434 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.616979 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.617981 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.618370 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.618443 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.618452 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.618646 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.618696 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:00.618937 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.615708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43937 (local address 127.0.200.129:55469)
0926 04:03:00.616070 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.616080 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.616143 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.616281 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.616282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.616284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.616365 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.616367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.617062 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.617063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.617883 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.617887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.618474 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.618475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.618560 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.618586 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.618641 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.618679 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":145,"thread_start_us":141,"threads_started":1}
I20260926 04:03:00.619148 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51d7714892945ebbe92aa61321e47df" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b" is_pre_election: true
I20260926 04:03:00.619191 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51d7714892945ebbe92aa61321e47df" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233" is_pre_election: true
I20260926 04:03:00.619320 2838 raft_consensus.cc:2454] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:00.619320 2978 raft_consensus.cc:2454] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:00.619542 2629 leader_election.cc:302] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:00.619725 3055 raft_consensus.cc:2781] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:00.619867 3055 raft_consensus.cc:482] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:00.619911 3055 raft_consensus.cc:3037] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.620637 3055 raft_consensus.cc:504] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.620815 3055 leader_election.cc:288] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Requested vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:00.621008 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51d7714892945ebbe92aa61321e47df" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233"
I20260926 04:03:00.621023 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d51d7714892945ebbe92aa61321e47df" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b"
I20260926 04:03:00.621125 2838 raft_consensus.cc:3037] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.621120 2978 raft_consensus.cc:3037] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.621966 2978 raft_consensus.cc:2454] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:00.621989 2838 raft_consensus.cc:2454] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:00.622602 2632 leader_election.cc:302] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 8ac6b3d44247404b8d005a5a7afb584b; no voters:
I20260926 04:03:00.622736 3055 raft_consensus.cc:2781] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:00.622893 3055 raft_consensus.cc:686] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Becoming Leader. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Running, Role: LEADER
I20260926 04:03:00.623003 3055 consensus_queue.cc:237] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.624022 2541 catalog_manager.cc:5495] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129). New cstate: current_term: 1 leader_uuid: "1284e3967f7e4f75befda8e10c79f4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } health_report { overall_health: UNKNOWN } } }
W20260926 04:03:00.630151 2885 tablet.cc:2369] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 04:03:00 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:00.667976 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.659583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45621 (local address 127.0.0.1:44780)
0926 04:03:00.659795 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.659808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.659902 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.660068 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.660068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.660509 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:00.660589 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.660593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.661591 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.661592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.662506 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.662509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.662930 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.662932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.662986 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.663534 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:00.663539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:00.664965 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:00.666858 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.666869 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.666872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.667058 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.667287 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:00.667288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:00.667289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:00.667349 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:00.667676 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:00.667683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:00.667729 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.667799 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.667899 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":129,"threads_started":1}
I20260926 04:03:00.668121 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.659612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.0.200.190:45621)
0926 04:03:00.659658 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.659663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.659874 (+ 211us) server_negotiation.cc:407] Connection header received
0926 04:03:00.659973 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.659974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.659984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.660064 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:00.660660 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.661494 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.662653 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.662838 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.662898 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.665078 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:00.665079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:00.665079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:00.665086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:00.666727 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.667153 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.667154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.667154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.667206 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:00.667415 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:00.667417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:00.667417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:00.667533 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:00.667620 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.667856 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.667961 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":20}
I20260926 04:03:00.670122 2541 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:44780:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 04:03:00.670318 2541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:00.672093 2678 tablet_service.cc:1467] Processing CreateTablet for tablet c2570e0f74c94143969525c02ed6bbe0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea4516f5322942a18536ef02a1c7b285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:00.672088 2818 tablet_service.cc:1467] Processing CreateTablet for tablet c2570e0f74c94143969525c02ed6bbe0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea4516f5322942a18536ef02a1c7b285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:00.672255 2958 tablet_service.cc:1467] Processing CreateTablet for tablet c2570e0f74c94143969525c02ed6bbe0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea4516f5322942a18536ef02a1c7b285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:00.672309 2678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2570e0f74c94143969525c02ed6bbe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.672322 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2570e0f74c94143969525c02ed6bbe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.672396 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2570e0f74c94143969525c02ed6bbe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:00.673579 3048 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: Bootstrap starting.
I20260926 04:03:00.673823 3051 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: Bootstrap starting.
I20260926 04:03:00.673844 3047 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: Bootstrap starting.
I20260926 04:03:00.674227 3051 tablet_bootstrap.cc:654] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.674360 3048 tablet_bootstrap.cc:654] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.674391 3047 tablet_bootstrap.cc:654] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:00.674892 3051 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: No bootstrap required, opened a new log
I20260926 04:03:00.674944 3051 ts_tablet_manager.cc:1397] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:00.675107 3051 raft_consensus.cc:348] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.675205 3051 raft_consensus.cc:374] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.675232 3051 raft_consensus.cc:729] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.675300 3051 consensus_queue.cc:260] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.675400 3051 ts_tablet_manager.cc:1428] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.675469 3055 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:00.675576 3055 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:00.675881 3048 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: No bootstrap required, opened a new log
I20260926 04:03:00.675940 3048 ts_tablet_manager.cc:1397] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:00.676096 3048 raft_consensus.cc:348] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.676173 3048 raft_consensus.cc:374] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.676198 3048 raft_consensus.cc:729] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266115e0ea70411e89992561b8f90233, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.676219 3047 tablet_bootstrap.cc:492] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: No bootstrap required, opened a new log
I20260926 04:03:00.676239 3048 consensus_queue.cc:260] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.676268 3047 ts_tablet_manager.cc:1397] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:00.676343 3048 ts_tablet_manager.cc:1428] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:00.676410 3047 raft_consensus.cc:348] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.676481 3047 raft_consensus.cc:374] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:00.676502 3047 raft_consensus.cc:729] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Initialized, Role: FOLLOWER
I20260926 04:03:00.676493 3052 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:00.676551 3047 consensus_queue.cc:260] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.676584 3052 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:00.676652 3047 ts_tablet_manager.cc:1428] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:00.676700 3053 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:00.676796 3053 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 04:03:00.712517 2745 tablet.cc:2369] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:03:00.792817 3025 tablet.cc:2369] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:00.971685 3053 raft_consensus.cc:482] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:00.971784 3053 raft_consensus.cc:504] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.972100 3053 leader_election.cc:288] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445), 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937)
I20260926 04:03:00.976155 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.972602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:37987 (local address 127.0.200.130:43937)
0926 04:03:00.972655 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.972657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.972792 (+ 135us) server_negotiation.cc:407] Connection header received
0926 04:03:00.972879 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.972881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.972891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.972950 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.973164 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.973883 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.975165 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.975724 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.975798 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.975807 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.976028 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.976076 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 04:03:00.976171 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.972529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43937 (local address 127.0.200.131:37987)
0926 04:03:00.972716 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.972726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.972790 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.973007 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.973008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.973011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.973094 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.973097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.973998 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.973999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.975071 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.975074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.975837 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.975839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.975927 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.975932 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.975999 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.976051 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260926 04:03:00.976333 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.972410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37445 (local address 127.0.200.131:58721)
0926 04:03:00.972502 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:00.972516 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:00.972582 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:00.972960 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:00.972961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:00.972963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:00.973043 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.973045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.973981 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.973983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.975319 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:00.975322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.976075 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.976076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:00.976159 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.976164 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:00.976211 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 04:03:00.976253 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 04:03:00.976445 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2570e0f74c94143969525c02ed6bbe0" candidate_uuid: "8ac6b3d44247404b8d005a5a7afb584b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233" is_pre_election: true
I20260926 04:03:00.976536 2838 raft_consensus.cc:2454] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac6b3d44247404b8d005a5a7afb584b in term 0.
I20260926 04:03:00.976750 2909 leader_election.cc:302] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266115e0ea70411e89992561b8f90233, 8ac6b3d44247404b8d005a5a7afb584b; no voters:
I20260926 04:03:00.976894 3053 raft_consensus.cc:2781] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:00.976940 3053 raft_consensus.cc:482] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:00.976972 3053 raft_consensus.cc:3037] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.977056 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:00.972768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:58721 (local address 127.0.200.129:37445)
0926 04:03:00.972815 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:00.972817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:00.972826 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:00.972855 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:00.972856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:00.972866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:00.972956 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:00.973130 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.973890 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.975429 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:00.975947 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:00.976039 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:00.976045 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:00.976923 (+ 878us) server_negotiation.cc:299] Negotiation successful
0926 04:03:00.976979 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 04:03:00.977298 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2570e0f74c94143969525c02ed6bbe0" candidate_uuid: "8ac6b3d44247404b8d005a5a7afb584b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1284e3967f7e4f75befda8e10c79f4ae" is_pre_election: true
I20260926 04:03:00.977412 2698 raft_consensus.cc:2454] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac6b3d44247404b8d005a5a7afb584b in term 0.
I20260926 04:03:00.977854 3053 raft_consensus.cc:504] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.978006 3053 leader_election.cc:288] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [CANDIDATE]: Term 1 election: Requested vote from peers 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445), 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937)
I20260926 04:03:00.978220 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2570e0f74c94143969525c02ed6bbe0" candidate_uuid: "8ac6b3d44247404b8d005a5a7afb584b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233"
I20260926 04:03:00.978219 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2570e0f74c94143969525c02ed6bbe0" candidate_uuid: "8ac6b3d44247404b8d005a5a7afb584b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1284e3967f7e4f75befda8e10c79f4ae"
I20260926 04:03:00.978297 2698 raft_consensus.cc:3037] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.978297 2838 raft_consensus.cc:3037] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:00.979050 2838 raft_consensus.cc:2454] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac6b3d44247404b8d005a5a7afb584b in term 1.
I20260926 04:03:00.979111 2698 raft_consensus.cc:2454] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac6b3d44247404b8d005a5a7afb584b in term 1.
I20260926 04:03:00.979245 2909 leader_election.cc:302] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 8ac6b3d44247404b8d005a5a7afb584b; no voters:
I20260926 04:03:00.979363 3053 raft_consensus.cc:2781] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:00.979506 3053 raft_consensus.cc:686] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 LEADER]: Becoming Leader. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Running, Role: LEADER
I20260926 04:03:00.979615 3053 consensus_queue.cc:237] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:00.980118 3069 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: TxnStatusTablet state changed. Reason: New leader 8ac6b3d44247404b8d005a5a7afb584b. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:00.980238 3069 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:00.980433 3071 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 04:03:00.980575 3071 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 04:03:00.980541 2540 catalog_manager.cc:5495] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131). New cstate: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:01.030819 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.028349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.0.200.131:44091)
0926 04:03:01.028405 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.028407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.028416 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:01.028489 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.028490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.028508 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.028593 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.028722 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.029195 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.029962 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.030155 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.030233 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.030420 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.030524 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.030669 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.030713 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.030734 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 04:03:01.030826 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.028280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.0.1:48170)
0926 04:03:01.028346 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.028358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.028409 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.028596 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.028597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.028598 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:01.028648 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.028651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.029283 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.029284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.029853 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.029855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.030231 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.030232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.030296 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.030314 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.030647 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.030657 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.030706 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.030756 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 04:03:01.032671 2838 raft_consensus.cc:1261] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Refusing update from remote peer 8ac6b3d44247404b8d005a5a7afb584b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.032698 2698 raft_consensus.cc:1261] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Refusing update from remote peer 8ac6b3d44247404b8d005a5a7afb584b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.032966 3053 consensus_queue.cc:1035] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [LEADER]: Connected to new peer: Peer: permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.033169 3069 consensus_queue.cc:1035] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.034163 3052 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: TxnStatusTablet state changed. Reason: New leader 8ac6b3d44247404b8d005a5a7afb584b. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.034273 3052 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:01.035235 3055 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: TxnStatusTablet state changed. Reason: New leader 8ac6b3d44247404b8d005a5a7afb584b. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.035341 3055 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:01.036057 3069 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.036147 3069 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:01.036104 3079 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.036232 3079 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 8ac6b3d44247404b8d005a5a7afb584b: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:01.036697 3052 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.036780 3052 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 266115e0ea70411e89992561b8f90233: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:01.037249 3055 tablet_replica.cc:439] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ac6b3d44247404b8d005a5a7afb584b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } } }
I20260926 04:03:01.037287 3075 mvcc.cc:205] Tried to move back new op lower bound from 7333459480706756608 to 7333459480492843008. Current Snapshot: MvccSnapshot[applied={T|T < 7333459480706756608}]
I20260926 04:03:01.037348 3055 tablet_replica.cc:442] T c2570e0f74c94143969525c02ed6bbe0 P 1284e3967f7e4f75befda8e10c79f4ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:01.037499 3077 mvcc.cc:205] Tried to move back new op lower bound from 7333459480706756608 to 7333459480492843008. Current Snapshot: MvccSnapshot[applied={T|T < 7333459480706756608}]
I20260926 04:03:01.041229 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.038785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.0.200.129:37445)
0926 04:03:01.038825 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.038827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.038834 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:03:01.038863 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.038864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.038881 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.038985 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.039143 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.039621 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.040407 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.040622 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.040701 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.040885 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.040998 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.041094 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.041122 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.041145 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:03:01.046907 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.043963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:40335 (local address 127.0.200.131:44091)
0926 04:03:01.044009 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.044011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.044108 (+ 97us) server_negotiation.cc:407] Connection header received
0926 04:03:01.044161 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.044162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.044172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.044229 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.044345 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.045084 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.045949 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.046239 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.046318 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.046641 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.046679 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.046706 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.046807 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.046827 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:01.047269 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.043888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.200.129:40335)
0926 04:03:01.044031 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.044042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.044105 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.044231 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.044231 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.044232 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:01.044287 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.044288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.045185 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.045186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.045858 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.045860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.046377 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.046379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.046452 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.046470 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.046745 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.046747 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.047126 (+ 379us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.047176 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260926 04:03:01.049995 2838 raft_consensus.cc:1261] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.050081 2978 raft_consensus.cc:1261] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.050371 3055 consensus_queue.cc:1035] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.050530 3057 consensus_queue.cc:1035] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.056149 3074 mvcc.cc:205] Tried to move back new op lower bound from 7333459480778358784 to 7333459479032463360. Current Snapshot: MvccSnapshot[applied={T|T < 7333459480778358784}]
I20260926 04:03:01.060784 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.057483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:43123 (local address 127.0.200.129:37445)
0926 04:03:01.057605 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.057607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.057701 (+ 94us) server_negotiation.cc:407] Connection header received
0926 04:03:01.057736 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.057737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.057750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.057812 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.058020 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.058753 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.059965 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.060256 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.060380 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.060535 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.060574 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.060640 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.060675 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.060700 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 04:03:01.060863 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.057406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37445 (local address 127.0.200.131:43123)
0926 04:03:01.057617 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.057629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.057690 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.057865 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.057866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.057868 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:01.057933 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.057935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.058851 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.058852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.059730 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.059733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.060368 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.060369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.060428 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.060444 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.060638 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.060639 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.060670 (+ 31us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.060709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260926 04:03:01.071019 2540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:01.071154 2540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:01.072033 2540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Sending DeleteTablet for 3 replicas of tablet d51d7714892945ebbe92aa61321e47df
I20260926 04:03:01.072396 2818 tablet_service.cc:1514] Processing DeleteTablet for tablet d51d7714892945ebbe92aa61321e47df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:01 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55466
I20260926 04:03:01.072700 3094 tablet_replica.cc:330] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: stopping tablet replica
I20260926 04:03:01.072826 3094 raft_consensus.cc:2229] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:01.072805 2958 tablet_service.cc:1514] Processing DeleteTablet for tablet d51d7714892945ebbe92aa61321e47df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:01 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:48160
I20260926 04:03:01.072887 3094 raft_consensus.cc:2258] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:01.073199 2678 tablet_service.cc:1514] Processing DeleteTablet for tablet d51d7714892945ebbe92aa61321e47df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:01 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55076
I20260926 04:03:01.073360 3094 ts_tablet_manager.cc:1905] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:01.073913 2540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:01.074064 2540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:01.075214 3094 ts_tablet_manager.cc:1918] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:01.075284 3094 log.cc:1196] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/d51d7714892945ebbe92aa61321e47df
I20260926 04:03:01.075539 3094 ts_tablet_manager.cc:1939] T d51d7714892945ebbe92aa61321e47df P 266115e0ea70411e89992561b8f90233: Deleting consensus metadata
I20260926 04:03:01.075605 2818 tablet_service.cc:1467] Processing CreateTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 (DEFAULT_TABLE table=test-table [id=3a1ebbd2759a403bbec018273b30a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:01.075757 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6550814dd4c4b29b59ebdef50a3bb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:01.076170 2525 catalog_manager.cc:4841] TS 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937): tablet d51d7714892945ebbe92aa61321e47df (table test-table [id=e2c0ef15120346fb81b836c2ce87e18e]) successfully deleted
I20260926 04:03:01.076959 3048 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Bootstrap starting.
I20260926 04:03:01.077174 2678 tablet_service.cc:1467] Processing CreateTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 (DEFAULT_TABLE table=test-table [id=3a1ebbd2759a403bbec018273b30a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:01.077288 2678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6550814dd4c4b29b59ebdef50a3bb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:01.077369 3048 tablet_bootstrap.cc:654] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:01.077983 3048 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: No bootstrap required, opened a new log
I20260926 04:03:01.078025 3048 ts_tablet_manager.cc:1397] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:01.078176 3048 raft_consensus.cc:348] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.078244 3048 raft_consensus.cc:374] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:01.078267 3048 raft_consensus.cc:729] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266115e0ea70411e89992561b8f90233, State: Initialized, Role: FOLLOWER
I20260926 04:03:01.078315 3048 consensus_queue.cc:260] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.078400 3048 ts_tablet_manager.cc:1428] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:01.078627 3051 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Bootstrap starting.
I20260926 04:03:01.079063 3051 tablet_bootstrap.cc:654] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:01.079561 2958 tablet_service.cc:1467] Processing CreateTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 (DEFAULT_TABLE table=test-table [id=3a1ebbd2759a403bbec018273b30a945]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:01.079674 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6550814dd4c4b29b59ebdef50a3bb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:01.079764 3051 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: No bootstrap required, opened a new log
I20260926 04:03:01.079809 3051 ts_tablet_manager.cc:1397] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:01.079944 3051 raft_consensus.cc:348] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.080018 3051 raft_consensus.cc:374] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:01.080039 3051 raft_consensus.cc:729] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Initialized, Role: FOLLOWER
I20260926 04:03:01.080082 3051 consensus_queue.cc:260] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.080163 3051 ts_tablet_manager.cc:1428] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:01.081131 3047 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Bootstrap starting.
I20260926 04:03:01.081538 3047 tablet_bootstrap.cc:654] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:01.082648 3096 tablet_replica.cc:330] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: stopping tablet replica
I20260926 04:03:01.082779 3096 raft_consensus.cc:2229] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:01.082883 3096 raft_consensus.cc:2258] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:01.083062 3047 tablet_bootstrap.cc:492] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: No bootstrap required, opened a new log
I20260926 04:03:01.083110 3047 ts_tablet_manager.cc:1397] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:01.083259 3047 raft_consensus.cc:348] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.083348 3047 raft_consensus.cc:374] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:01.083369 3047 raft_consensus.cc:729] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Initialized, Role: FOLLOWER
I20260926 04:03:01.083406 3096 ts_tablet_manager.cc:1905] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:01.083424 3047 consensus_queue.cc:260] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.083520 3047 ts_tablet_manager.cc:1428] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:01.084196 3095 tablet_replica.cc:330] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: stopping tablet replica
I20260926 04:03:01.084316 3095 raft_consensus.cc:2229] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:01.084383 3095 raft_consensus.cc:2258] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:01.084827 3095 ts_tablet_manager.cc:1905] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:01.085426 3096 ts_tablet_manager.cc:1918] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:01.085546 3096 log.cc:1196] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/d51d7714892945ebbe92aa61321e47df
I20260926 04:03:01.086208 3096 ts_tablet_manager.cc:1939] T d51d7714892945ebbe92aa61321e47df P 1284e3967f7e4f75befda8e10c79f4ae: Deleting consensus metadata
I20260926 04:03:01.086438 3095 ts_tablet_manager.cc:1918] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:01.086603 3095 log.cc:1196] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/d51d7714892945ebbe92aa61321e47df
I20260926 04:03:01.086740 2525 catalog_manager.cc:4841] TS 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445): tablet d51d7714892945ebbe92aa61321e47df (table test-table [id=e2c0ef15120346fb81b836c2ce87e18e]) successfully deleted
I20260926 04:03:01.087132 3095 ts_tablet_manager.cc:1939] T d51d7714892945ebbe92aa61321e47df P 8ac6b3d44247404b8d005a5a7afb584b: Deleting consensus metadata
I20260926 04:03:01.087596 2528 catalog_manager.cc:4841] TS 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091): tablet d51d7714892945ebbe92aa61321e47df (table test-table [id=e2c0ef15120346fb81b836c2ce87e18e]) successfully deleted
I20260926 04:03:01.209798 3052 raft_consensus.cc:482] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:01.209911 3052 raft_consensus.cc:504] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.210333 3052 leader_election.cc:288] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:01.213940 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.210524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37445 (local address 127.0.200.130:53083)
0926 04:03:01.210768 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.210778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.210846 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.211101 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.211102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.211104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:01.211173 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.211175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.212020 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.212021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.212873 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.212875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.213636 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.213637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.213719 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.213724 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.213778 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.213818 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":87,"threads_started":1}
I20260926 04:03:01.213937 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.210650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:53083 (local address 127.0.200.129:37445)
0926 04:03:01.210699 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.210701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.210919 (+ 218us) server_negotiation.cc:407] Connection header received
0926 04:03:01.210953 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.210954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.210964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.211036 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:01.211261 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.211950 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.212998 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.213542 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.213618 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.213627 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.213808 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.213842 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:01.214226 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6550814dd4c4b29b59ebdef50a3bb86" candidate_uuid: "266115e0ea70411e89992561b8f90233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1284e3967f7e4f75befda8e10c79f4ae" is_pre_election: true
I20260926 04:03:01.214329 2698 raft_consensus.cc:2454] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266115e0ea70411e89992561b8f90233 in term 0.
I20260926 04:03:01.214406 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.210620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:53413 (local address 127.0.200.131:44091)
0926 04:03:01.210676 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.210678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.210746 (+ 68us) server_negotiation.cc:407] Connection header received
0926 04:03:01.210820 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.210822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.210836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.210896 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:01.211108 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.211891 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.213331 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.213864 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.213929 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.213936 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.214300 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.214341 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:03:01.214593 2769 leader_election.cc:302] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:01.214713 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.210517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.200.130:53413)
0926 04:03:01.210678 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.210691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.210767 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.210941 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.210942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.210945 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:01.211033 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.211035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.212002 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.212003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.213232 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.213235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.213969 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.213970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.214201 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.214205 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.214264 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.214583 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":19,"thread_start_us":120,"threads_started":1}
I20260926 04:03:01.214769 3052 raft_consensus.cc:2781] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:01.214815 3052 raft_consensus.cc:482] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:01.214845 3052 raft_consensus.cc:3037] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:01.214975 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6550814dd4c4b29b59ebdef50a3bb86" candidate_uuid: "266115e0ea70411e89992561b8f90233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b" is_pre_election: true
I20260926 04:03:01.215072 2978 raft_consensus.cc:2454] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266115e0ea70411e89992561b8f90233 in term 0.
I20260926 04:03:01.216080 3052 raft_consensus.cc:504] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.216241 3052 leader_election.cc:288] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [CANDIDATE]: Term 1 election: Requested vote from peers 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:01.216413 2698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6550814dd4c4b29b59ebdef50a3bb86" candidate_uuid: "266115e0ea70411e89992561b8f90233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1284e3967f7e4f75befda8e10c79f4ae"
I20260926 04:03:01.216521 2698 raft_consensus.cc:3037] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:01.217263 2698 raft_consensus.cc:2454] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266115e0ea70411e89992561b8f90233 in term 1.
I20260926 04:03:01.217412 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6550814dd4c4b29b59ebdef50a3bb86" candidate_uuid: "266115e0ea70411e89992561b8f90233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b"
I20260926 04:03:01.217474 2769 leader_election.cc:302] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:01.217514 2978 raft_consensus.cc:3037] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:01.217566 3052 raft_consensus.cc:2781] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:01.217803 3052 raft_consensus.cc:686] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 1 LEADER]: Becoming Leader. State: Replica: 266115e0ea70411e89992561b8f90233, State: Running, Role: LEADER
I20260926 04:03:01.217887 3052 consensus_queue.cc:237] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:01.218278 2978 raft_consensus.cc:2454] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266115e0ea70411e89992561b8f90233 in term 1.
I20260926 04:03:01.218652 2540 catalog_manager.cc:5495] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 reported cstate change: term changed from 0 to 1, leader changed from <none> to 266115e0ea70411e89992561b8f90233 (127.0.200.130). New cstate: current_term: 1 leader_uuid: "266115e0ea70411e89992561b8f90233" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:01.237336 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.234990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.0.200.130:43937)
0926 04:03:01.235065 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.235067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.235073 (+ 6us) server_negotiation.cc:407] Connection header received
0926 04:03:01.235104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.235105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.235127 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.235193 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.235355 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.235820 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.236545 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.236726 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.236784 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.237004 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.237109 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.237169 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.237255 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.237275 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 04:03:01.242787 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.240305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:53319 (local address 127.0.200.131:44091)
0926 04:03:01.240393 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.240395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.240472 (+ 77us) server_negotiation.cc:407] Connection header received
0926 04:03:01.240542 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.240542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.240559 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.240600 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.240829 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.241287 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.242062 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.242245 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.242305 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.242513 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.242538 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.242652 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.242711 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.242728 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 04:03:01.243062 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.240231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44091 (local address 127.0.200.130:53319)
0926 04:03:01.240404 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.240419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.240479 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.240640 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.240641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.240643 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:01.240744 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.240746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.241359 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.241359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.241960 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.241961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.242361 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.242362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.242412 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.242425 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.242663 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.242664 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.242697 (+ 33us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.242734 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260926 04:03:01.245252 2698 raft_consensus.cc:1261] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Refusing update from remote peer 266115e0ea70411e89992561b8f90233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.245293 2978 raft_consensus.cc:1261] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Refusing update from remote peer 266115e0ea70411e89992561b8f90233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:01.245546 3052 consensus_queue.cc:1035] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.245698 3102 consensus_queue.cc:1035] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:01.253363 3043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.250947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:53279 (local address 127.0.200.130:43937)
0926 04:03:01.250982 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:01.250983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:01.251008 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:03:01.251072 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:01.251073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:01.251082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:01.251136 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:01.251278 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.251822 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.252576 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.252888 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.252952 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.253112 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.253151 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.253204 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:01.253264 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 04:03:01.253285 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 04:03:01.253372 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:01.250915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43937 (local address 127.0.200.131:53279)
0926 04:03:01.250962 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:01.250975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:01.251020 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:01.251140 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:01.251140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:01.251141 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:01.251187 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.251189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.251936 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.251937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.252488 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:01.252490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:01.252970 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:01.252971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:01.253031 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:01.253038 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:01.253206 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:01.253209 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:01.253252 (+ 43us) client_negotiation.cc:240] Negotiation successful
0926 04:03:01.253306 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 04:03:01.253837 3075 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 04:03:02.773495 2540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:02.773627 2540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:02.774619 2540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Sending DeleteTablet for 3 replicas of tablet e6550814dd4c4b29b59ebdef50a3bb86
I20260926 04:03:02.774989 2818 tablet_service.cc:1514] Processing DeleteTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55466
I20260926 04:03:02.775077 2958 tablet_service.cc:1514] Processing DeleteTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:48160
I20260926 04:03:02.775036 2678 tablet_service.cc:1514] Processing DeleteTablet for tablet e6550814dd4c4b29b59ebdef50a3bb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55076
I20260926 04:03:02.775319 3135 tablet_replica.cc:330] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: stopping tablet replica
I20260926 04:03:02.775327 3136 tablet_replica.cc:330] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: stopping tablet replica
I20260926 04:03:02.775626 3136 raft_consensus.cc:2229] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:02.775616 3135 raft_consensus.cc:2229] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:02.775715 3135 raft_consensus.cc:2258] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.775715 3136 raft_consensus.cc:2258] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.776011 3136 ts_tablet_manager.cc:1905] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:02.776032 3135 ts_tablet_manager.cc:1905] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:02.777971 3136 ts_tablet_manager.cc:1918] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 04:03:02.777980 3134 tablet_replica.cc:330] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: stopping tablet replica
I20260926 04:03:02.778035 3136 log.cc:1196] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/e6550814dd4c4b29b59ebdef50a3bb86
I20260926 04:03:02.778045 3134 raft_consensus.cc:2229] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:02.778170 3134 raft_consensus.cc:2258] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.778283 3136 ts_tablet_manager.cc:1939] T e6550814dd4c4b29b59ebdef50a3bb86 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting consensus metadata
I20260926 04:03:02.778210 3135 ts_tablet_manager.cc:1918] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 04:03:02.778362 3135 log.cc:1196] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/e6550814dd4c4b29b59ebdef50a3bb86
I20260926 04:03:02.778403 3134 ts_tablet_manager.cc:1905] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:02.778618 3135 ts_tablet_manager.cc:1939] T e6550814dd4c4b29b59ebdef50a3bb86 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting consensus metadata
I20260926 04:03:02.779291 2528 catalog_manager.cc:4841] TS 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091): tablet e6550814dd4c4b29b59ebdef50a3bb86 (table test-table [id=3a1ebbd2759a403bbec018273b30a945]) successfully deleted
I20260926 04:03:02.779625 2525 catalog_manager.cc:4841] TS 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445): tablet e6550814dd4c4b29b59ebdef50a3bb86 (table test-table [id=3a1ebbd2759a403bbec018273b30a945]) successfully deleted
I20260926 04:03:02.780150 3134 ts_tablet_manager.cc:1918] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 04:03:02.780213 3134 log.cc:1196] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/e6550814dd4c4b29b59ebdef50a3bb86
I20260926 04:03:02.780431 3134 ts_tablet_manager.cc:1939] T e6550814dd4c4b29b59ebdef50a3bb86 P 266115e0ea70411e89992561b8f90233: Deleting consensus metadata
I20260926 04:03:02.781013 2525 catalog_manager.cc:4841] TS 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937): tablet e6550814dd4c4b29b59ebdef50a3bb86 (table test-table [id=3a1ebbd2759a403bbec018273b30a945]) successfully deleted
Sep 26 04:03:02 dist-test-slave-j8c1 krb5kdc[2496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 04:03:02 dist-test-slave-j8c1 krb5kdc[2496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:02.816253 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:02.794144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.0.200.190:45621)
0926 04:03:02.794337 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:02.794339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:02.794358 (+ 19us) server_negotiation.cc:407] Connection header received
0926 04:03:02.794484 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:02.794486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:02.794509 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:02.794700 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:02.795390 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:02.795923 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:02.796806 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:02.797058 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:02.797139 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:02.813148 (+ 16009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:02.813153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:02.813153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:02.813168 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:02.814890 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:02.815355 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:02.815357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:02.815357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:02.815421 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:02.815644 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:02.815645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:02.815645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:02.815782 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:02.815900 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:02.816045 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 04:03:02.816136 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":131,"threads_started":1}
I20260926 04:03:02.818802 2540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:02.818996 2540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:02.820760 2678 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 (DEFAULT_TABLE table=test-table [id=e8f49538845143b08483474a5a2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.820760 2818 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 (DEFAULT_TABLE table=test-table [id=e8f49538845143b08483474a5a2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.820916 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8bff8f58ca403da91681bbb73262e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.820917 2678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8bff8f58ca403da91681bbb73262e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.822350 2958 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 (DEFAULT_TABLE table=test-table [id=e8f49538845143b08483474a5a2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.822355 3145 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Bootstrap starting.
I20260926 04:03:02.822482 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8bff8f58ca403da91681bbb73262e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.822856 3145 tablet_bootstrap.cc:654] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.823208 3146 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Bootstrap starting.
I20260926 04:03:02.823565 3145 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: No bootstrap required, opened a new log
I20260926 04:03:02.823628 3146 tablet_bootstrap.cc:654] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.823637 3145 ts_tablet_manager.cc:1397] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:02.823771 3145 raft_consensus.cc:348] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.823837 3148 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Bootstrap starting.
I20260926 04:03:02.823853 3145 raft_consensus.cc:374] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.823879 3145 raft_consensus.cc:729] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.823947 3145 consensus_queue.cc:260] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.824142 3145 ts_tablet_manager.cc:1428] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:02.824187 3148 tablet_bootstrap.cc:654] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.824723 3146 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: No bootstrap required, opened a new log
I20260926 04:03:02.824779 3146 ts_tablet_manager.cc:1397] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:02.824899 3148 tablet_bootstrap.cc:492] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: No bootstrap required, opened a new log
I20260926 04:03:02.824940 3148 ts_tablet_manager.cc:1397] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:02.824920 3146 raft_consensus.cc:348] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.824994 3146 raft_consensus.cc:374] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.825019 3146 raft_consensus.cc:729] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266115e0ea70411e89992561b8f90233, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.825047 3148 raft_consensus.cc:348] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.825120 3148 raft_consensus.cc:374] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.825088 3146 consensus_queue.cc:260] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.825150 3148 raft_consensus.cc:729] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.825203 3148 consensus_queue.cc:260] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.825225 3146 ts_tablet_manager.cc:1428] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:02.825321 3148 ts_tablet_manager.cc:1428] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:02.851287 3149 raft_consensus.cc:482] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:02.851400 3149 raft_consensus.cc:504] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.851580 3149 leader_election.cc:288] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:02.851845 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8bff8f58ca403da91681bbb73262e7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233" is_pre_election: true
I20260926 04:03:02.851948 2838 raft_consensus.cc:2454] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:02.851895 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8bff8f58ca403da91681bbb73262e7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b" is_pre_election: true
I20260926 04:03:02.852005 2978 raft_consensus.cc:2454] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:02.852144 2629 leader_election.cc:302] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:02.852246 3149 raft_consensus.cc:2781] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:02.852284 3149 raft_consensus.cc:482] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:02.852300 3149 raft_consensus.cc:3037] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.852959 3149 raft_consensus.cc:504] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.853102 3149 leader_election.cc:288] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Requested vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:02.853309 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8bff8f58ca403da91681bbb73262e7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233"
I20260926 04:03:02.853369 2838 raft_consensus.cc:3037] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.853586 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8bff8f58ca403da91681bbb73262e7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b"
I20260926 04:03:02.853670 2978 raft_consensus.cc:3037] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.854159 2838 raft_consensus.cc:2454] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:02.854393 2629 leader_election.cc:302] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:02.854483 2978 raft_consensus.cc:2454] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:02.854518 3149 raft_consensus.cc:2781] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:02.854758 3149 raft_consensus.cc:686] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Becoming Leader. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Running, Role: LEADER
I20260926 04:03:02.854836 3149 consensus_queue.cc:237] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.855455 2540 catalog_manager.cc:5495] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129). New cstate: current_term: 1 leader_uuid: "1284e3967f7e4f75befda8e10c79f4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:02.864931 3155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:02.862153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.0.200.129:37445)
0926 04:03:02.862321 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:02.862322 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:02.862344 (+ 22us) server_negotiation.cc:407] Connection header received
0926 04:03:02.862364 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:02.862364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:02.862373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:02.862462 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:02.862680 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:02.863246 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:02.864110 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:02.864357 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:02.864447 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:02.864620 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:02.864656 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:02.864733 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:02.864789 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 04:03:02.864814 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260926 04:03:02.867314 2838 raft_consensus.cc:1261] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:02.867430 2978 raft_consensus.cc:1261] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:02.867579 3152 consensus_queue.cc:1035] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:02.867694 3149 consensus_queue.cc:1035] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:02.877326 2541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:02.877447 2541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:02.878201 2541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Sending DeleteTablet for 3 replicas of tablet 6a8bff8f58ca403da91681bbb73262e7
I20260926 04:03:02.878470 2678 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55076
I20260926 04:03:02.878515 2958 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:48160
I20260926 04:03:02.878624 3135 tablet_replica.cc:330] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: stopping tablet replica
I20260926 04:03:02.878605 2818 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a8bff8f58ca403da91681bbb73262e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:02 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55466
I20260926 04:03:02.878679 3135 raft_consensus.cc:2229] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:02.878687 3136 tablet_replica.cc:330] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: stopping tablet replica
I20260926 04:03:02.878729 3135 raft_consensus.cc:2258] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.878743 3136 raft_consensus.cc:2229] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:02.878795 3134 tablet_replica.cc:330] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: stopping tablet replica
I20260926 04:03:02.878806 3136 raft_consensus.cc:2258] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.878865 3134 raft_consensus.cc:2229] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:02.878897 3134 raft_consensus.cc:2258] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:02.879060 2541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 04:03:02.879137 3136 ts_tablet_manager.cc:1905] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 04:03:02.879215 2541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:02.879325 3135 ts_tablet_manager.cc:1905] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:02.879331 3134 ts_tablet_manager.cc:1905] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:02.880973 3136 ts_tablet_manager.cc:1918] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:02.880996 3134 ts_tablet_manager.cc:1918] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:02.881059 3134 log.cc:1196] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/6a8bff8f58ca403da91681bbb73262e7
I20260926 04:03:02.881060 3136 log.cc:1196] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/6a8bff8f58ca403da91681bbb73262e7
I20260926 04:03:02.881265 3135 ts_tablet_manager.cc:1918] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:02.881306 3134 ts_tablet_manager.cc:1939] T 6a8bff8f58ca403da91681bbb73262e7 P 266115e0ea70411e89992561b8f90233: Deleting consensus metadata
I20260926 04:03:02.881304 3136 ts_tablet_manager.cc:1939] T 6a8bff8f58ca403da91681bbb73262e7 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting consensus metadata
I20260926 04:03:02.881322 3135 log.cc:1196] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/6a8bff8f58ca403da91681bbb73262e7
I20260926 04:03:02.881556 3135 ts_tablet_manager.cc:1939] T 6a8bff8f58ca403da91681bbb73262e7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting consensus metadata
I20260926 04:03:02.881811 2525 catalog_manager.cc:4841] TS 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937): tablet 6a8bff8f58ca403da91681bbb73262e7 (table test-table [id=e8f49538845143b08483474a5a2db152]) successfully deleted
I20260926 04:03:02.881841 2528 catalog_manager.cc:4841] TS 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091): tablet 6a8bff8f58ca403da91681bbb73262e7 (table test-table [id=e8f49538845143b08483474a5a2db152]) successfully deleted
I20260926 04:03:02.882088 2958 tablet_service.cc:1467] Processing CreateTablet for tablet 0abb9a4821c0435c8b14120130e030d7 (DEFAULT_TABLE table=test-table [id=60ddb73c8bea45eaa92904ebae3a918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.882138 2678 tablet_service.cc:1467] Processing CreateTablet for tablet 0abb9a4821c0435c8b14120130e030d7 (DEFAULT_TABLE table=test-table [id=60ddb73c8bea45eaa92904ebae3a918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.882198 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abb9a4821c0435c8b14120130e030d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.882294 2678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abb9a4821c0435c8b14120130e030d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.882642 2818 tablet_service.cc:1467] Processing CreateTablet for tablet 0abb9a4821c0435c8b14120130e030d7 (DEFAULT_TABLE table=test-table [id=60ddb73c8bea45eaa92904ebae3a918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:02.882745 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0abb9a4821c0435c8b14120130e030d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:02.883602 3145 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: Bootstrap starting.
I20260926 04:03:02.883652 3148 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Bootstrap starting.
I20260926 04:03:02.883936 3145 tablet_bootstrap.cc:654] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.883951 3148 tablet_bootstrap.cc:654] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.884087 2525 catalog_manager.cc:4841] TS 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445): tablet 6a8bff8f58ca403da91681bbb73262e7 (table test-table [id=e8f49538845143b08483474a5a2db152]) successfully deleted
I20260926 04:03:02.884203 3146 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Bootstrap starting.
I20260926 04:03:02.884537 3146 tablet_bootstrap.cc:654] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:02.884677 3145 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: No bootstrap required, opened a new log
I20260926 04:03:02.884761 3145 ts_tablet_manager.cc:1397] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:02.884876 3145 raft_consensus.cc:348] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.884987 3145 raft_consensus.cc:374] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.885010 3145 raft_consensus.cc:729] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.885056 3145 consensus_queue.cc:260] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.885088 3146 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: No bootstrap required, opened a new log
I20260926 04:03:02.885128 3146 ts_tablet_manager.cc:1397] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:02.885155 3145 ts_tablet_manager.cc:1428] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:02.885226 3146 raft_consensus.cc:348] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.885293 3146 raft_consensus.cc:374] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.885313 3146 raft_consensus.cc:729] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266115e0ea70411e89992561b8f90233, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.885355 3146 consensus_queue.cc:260] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.885416 3148 tablet_bootstrap.cc:492] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: No bootstrap required, opened a new log
I20260926 04:03:02.885461 3148 ts_tablet_manager.cc:1397] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:02.885473 3146 ts_tablet_manager.cc:1428] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:02.885560 3148 raft_consensus.cc:348] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.885630 3148 raft_consensus.cc:374] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:02.885654 3148 raft_consensus.cc:729] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac6b3d44247404b8d005a5a7afb584b, State: Initialized, Role: FOLLOWER
I20260926 04:03:02.885694 3148 consensus_queue.cc:260] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.885802 3148 ts_tablet_manager.cc:1428] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:02.895587 3152 raft_consensus.cc:482] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:02.895699 3152 raft_consensus.cc:504] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.895867 3152 leader_election.cc:288] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:02.896037 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abb9a4821c0435c8b14120130e030d7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233" is_pre_election: true
I20260926 04:03:02.896133 2838 raft_consensus.cc:2454] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:02.896157 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abb9a4821c0435c8b14120130e030d7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b" is_pre_election: true
I20260926 04:03:02.896327 2978 raft_consensus.cc:2454] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 0.
I20260926 04:03:02.896418 2629 leader_election.cc:302] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 266115e0ea70411e89992561b8f90233; no voters:
I20260926 04:03:02.896520 3152 raft_consensus.cc:2781] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:02.896559 3152 raft_consensus.cc:482] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:02.896576 3152 raft_consensus.cc:3037] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.897604 3152 raft_consensus.cc:504] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.897773 3152 leader_election.cc:288] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Requested vote from peers 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937), 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091)
I20260926 04:03:02.897958 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abb9a4821c0435c8b14120130e030d7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ac6b3d44247404b8d005a5a7afb584b"
I20260926 04:03:02.898015 2838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0abb9a4821c0435c8b14120130e030d7" candidate_uuid: "1284e3967f7e4f75befda8e10c79f4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266115e0ea70411e89992561b8f90233"
I20260926 04:03:02.898083 2978 raft_consensus.cc:3037] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.898094 2838 raft_consensus.cc:3037] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:02.899003 2978 raft_consensus.cc:2454] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:02.899017 2838 raft_consensus.cc:2454] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1284e3967f7e4f75befda8e10c79f4ae in term 1.
I20260926 04:03:02.899191 2632 leader_election.cc:302] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1284e3967f7e4f75befda8e10c79f4ae, 8ac6b3d44247404b8d005a5a7afb584b; no voters:
I20260926 04:03:02.899314 3152 raft_consensus.cc:2781] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:02.899379 3152 raft_consensus.cc:686] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Becoming Leader. State: Replica: 1284e3967f7e4f75befda8e10c79f4ae, State: Running, Role: LEADER
I20260926 04:03:02.899430 3152 consensus_queue.cc:237] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } }
I20260926 04:03:02.900101 2541 catalog_manager.cc:5495] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129). New cstate: current_term: 1 leader_uuid: "1284e3967f7e4f75befda8e10c79f4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1284e3967f7e4f75befda8e10c79f4ae" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:02.910580 2838 raft_consensus.cc:1261] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:02.910580 2978 raft_consensus.cc:1261] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Refusing update from remote peer 1284e3967f7e4f75befda8e10c79f4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:02.910861 3152 consensus_queue.cc:1035] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "266115e0ea70411e89992561b8f90233" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:02.911001 3159 consensus_queue.cc:1035] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ac6b3d44247404b8d005a5a7afb584b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 04:03:02.915774 3117 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 04:03:04.435315 2541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:04.435426 2541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:04.436568 2541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f06fd1ea364c4a68850b5783049f734a: Sending DeleteTablet for 3 replicas of tablet 0abb9a4821c0435c8b14120130e030d7
I20260926 04:03:04.436884 2678 tablet_service.cc:1514] Processing DeleteTablet for tablet 0abb9a4821c0435c8b14120130e030d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:04 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55076
I20260926 04:03:04.436936 2818 tablet_service.cc:1514] Processing DeleteTablet for tablet 0abb9a4821c0435c8b14120130e030d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:04 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55466
I20260926 04:03:04.437103 3189 tablet_replica.cc:330] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: stopping tablet replica
I20260926 04:03:04.437093 2957 tablet_service.cc:1514] Processing DeleteTablet for tablet 0abb9a4821c0435c8b14120130e030d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:04 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:48160
I20260926 04:03:04.437163 3189 raft_consensus.cc:2229] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:04.437247 3189 raft_consensus.cc:2258] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:04.437355 3188 tablet_replica.cc:330] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: stopping tablet replica
I20260926 04:03:04.437456 3188 raft_consensus.cc:2229] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:04.437538 3189 ts_tablet_manager.cc:1905] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:04.437554 3188 raft_consensus.cc:2258] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:04.437803 3188 ts_tablet_manager.cc:1905] T 0abb9a4821c0435c8b14120130e030d7 P 1284e3967f7e4f75befda8e10c79f4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:04.437973 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2613
I20260926 04:03:04.438947 3190 tablet_replica.cc:330] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: stopping tablet replica
I20260926 04:03:04.439011 3190 raft_consensus.cc:2229] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:04.439054 3190 raft_consensus.cc:2258] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:04.439303 3190 ts_tablet_manager.cc:1905] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:04.439893 3189 ts_tablet_manager.cc:1918] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 04:03:04.440018 3189 log.cc:1196] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/0abb9a4821c0435c8b14120130e030d7
I20260926 04:03:04.440315 3189 ts_tablet_manager.cc:1939] T 0abb9a4821c0435c8b14120130e030d7 P 266115e0ea70411e89992561b8f90233: Deleting consensus metadata
I20260926 04:03:04.440656 2525 catalog_manager.cc:4841] TS 266115e0ea70411e89992561b8f90233 (127.0.200.130:43937): tablet 0abb9a4821c0435c8b14120130e030d7 (table test-table [id=60ddb73c8bea45eaa92904ebae3a918e]) successfully deleted
I20260926 04:03:04.441069 3190 ts_tablet_manager.cc:1918] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 04:03:04.441119 3190 log.cc:1196] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/0abb9a4821c0435c8b14120130e030d7
I20260926 04:03:04.441284 3190 ts_tablet_manager.cc:1939] T 0abb9a4821c0435c8b14120130e030d7 P 8ac6b3d44247404b8d005a5a7afb584b: Deleting consensus metadata
I20260926 04:03:04.442018 2528 catalog_manager.cc:4841] TS 8ac6b3d44247404b8d005a5a7afb584b (127.0.200.131:44091): tablet 0abb9a4821c0435c8b14120130e030d7 (table test-table [id=60ddb73c8bea45eaa92904ebae3a918e]) successfully deleted
W20260926 04:03:04.444422 2525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:37445 (error 108)
I20260926 04:03:04.444659 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2753
I20260926 04:03:04.445715 3192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:04.444825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37445 (local address 127.0.0.1:55102)
0926 04:03:04.445635 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:04.445679 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:37445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":65,"threads_started":1}
W20260926 04:03:04.445830 2525 catalog_manager.cc:4584] TS 1284e3967f7e4f75befda8e10c79f4ae (127.0.200.129:37445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0abb9a4821c0435c8b14120130e030d7: Network error: Client connection negotiation failed: client connection to 127.0.200.129:37445: connect: Connection refused (error 111)
I20260926 04:03:04.451397 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2893
I20260926 04:03:04.457190 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 2509
2026-09-26T04:03:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5804 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:04 dist-test-slave-j8c1 krb5kdc[3195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:04 dist-test-slave-j8c1 krb5kdc[3195](info): set up 2 sockets
Sep 26 04:03:04 dist-test-slave-j8c1 krb5kdc[3195](info): commencing operation
krb5kdc: starting...
W20260926 04:03:05.518893 802 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:05 dist-test-slave-j8c1 krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:05.646431 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43063
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:05.730535 3208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:05.730792 3208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:05.730816 3208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:05.732129 3208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:05.732187 3208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:05.732203 3208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:05.732215 3208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:05.732227 3208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:05.733896 3208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:05.734179 3208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:05.736881 3208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:05.742127 3216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:05.742133 3214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:05.742141 3213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:05.742846 3208 server_base.cc:1034] running on GCE node
I20260926 04:03:05.743132 3208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:05.743551 3208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:05.744750 3208 hybrid_clock.cc:648] HybridClock initialized: now 1790395385744723 us; error 48 us; skew 500 ppm
Sep 26 04:03:05 dist-test-slave-j8c1 krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:05.747627 3208 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:05.748135 3208 webserver.cc:460] Webserver started at http://127.0.200.190:35697/ using document root <none> and password file <none>
I20260926 04:03:05.748409 3208 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:05.748457 3208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:05.748605 3208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:05.749707 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "d55cd9dfa35d47d284f3da177e21a735"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "13a7b85b47c10d7a6203301f8f6b2bb8"
server_key_iv: "9fdcc59c86d6195ebebc65b52e1958af"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.750072 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "d55cd9dfa35d47d284f3da177e21a735"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "13a7b85b47c10d7a6203301f8f6b2bb8"
server_key_iv: "9fdcc59c86d6195ebebc65b52e1958af"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.751714 3208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:05.752727 3223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:05.752983 3208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:05.753106 3208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "d55cd9dfa35d47d284f3da177e21a735"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "13a7b85b47c10d7a6203301f8f6b2bb8"
server_key_iv: "9fdcc59c86d6195ebebc65b52e1958af"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.753194 3208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:05.774816 3208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:05.777382 3208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:05.777588 3208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:05.782337 3208 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43063
I20260926 04:03:05.782434 3275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43063 every 8 connection(s)
I20260926 04:03:05.783022 3208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:05.783774 3276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:05.786195 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: Bootstrap starting.
I20260926 04:03:05.786906 3276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:05.787163 3276 log.cc:824] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:05.787937 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: No bootstrap required, opened a new log
I20260926 04:03:05.789531 3276 raft_consensus.cc:348] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } }
I20260926 04:03:05.789690 3276 raft_consensus.cc:374] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:05.789714 3276 raft_consensus.cc:729] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55cd9dfa35d47d284f3da177e21a735, State: Initialized, Role: FOLLOWER
I20260926 04:03:05.789882 3276 consensus_queue.cc:260] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } }
I20260926 04:03:05.789973 3276 raft_consensus.cc:388] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:05.789999 3276 raft_consensus.cc:482] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:05.790030 3276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:05.791043 3276 raft_consensus.cc:504] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } }
I20260926 04:03:05.791142 3276 leader_election.cc:302] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d55cd9dfa35d47d284f3da177e21a735; no voters:
I20260926 04:03:05.791312 3276 leader_election.cc:288] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:05.791615 3279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:05.791816 3279 raft_consensus.cc:686] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [term 1 LEADER]: Becoming Leader. State: Replica: d55cd9dfa35d47d284f3da177e21a735, State: Running, Role: LEADER
I20260926 04:03:05.791884 3276 sys_catalog.cc:564] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:05.791942 3279 consensus_queue.cc:237] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } }
I20260926 04:03:05.792237 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3208
I20260926 04:03:05.792318 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:05.792507 802 external_mini_cluster.cc:1428] Setting key 398d92716deb275048291a35a5410192
I20260926 04:03:05.793123 3280 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d55cd9dfa35d47d284f3da177e21a735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } } }
I20260926 04:03:05.793553 3281 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d55cd9dfa35d47d284f3da177e21a735. Latest consensus state: current_term: 1 leader_uuid: "d55cd9dfa35d47d284f3da177e21a735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55cd9dfa35d47d284f3da177e21a735" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43063 } } }
I20260926 04:03:05.793354 3280 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:05.793639 3281 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735 [sys.catalog]: This master's current role is: LEADER
W20260926 04:03:05.794121 3296 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:03:05.794189 3296 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:05.794256 3297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:05.794448 3297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:05.796125 3297 catalog_manager.cc:1347] Generated new cluster ID: 700de41f7a674490adbc0a08094adb49
I20260926 04:03:05.796181 3297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:05 dist-test-slave-j8c1 krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:05.805760 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:05.793223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55694 (local address 127.0.200.190:43063)
0926 04:03:05.793537 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:05.793549 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:05.793728 (+ 179us) server_negotiation.cc:407] Connection header received
0926 04:03:05.794048 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:05.794058 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:05.794128 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:05.794282 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:05.795140 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.796720 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.797897 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.798652 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.798772 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:05.801768 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:05.801773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:05.801774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:05.801788 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:05.803912 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:05.804365 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:05.804368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:05.804368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:05.804506 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:05.804729 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:05.804730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:05.804730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:05.805062 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:05.805219 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:05.805360 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 04:03:05.805515 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":58,"threads_started":1}
I20260926 04:03:05.813606 3297 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:05.814388 3297 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:05.823222 3297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d55cd9dfa35d47d284f3da177e21a735: Generated new TSK 0
I20260926 04:03:05.823508 3297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:05.854852 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:05.942462 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:05.942776 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:05.942803 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:05.944018 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:05.944058 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:05.944085 3302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:05.945638 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:05.945860 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:05.946061 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:05.951676 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:05.951675 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:05.951676 3310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:05.952067 3302 server_base.cc:1034] running on GCE node
I20260926 04:03:05.952527 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:05.952804 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:05.954049 3302 hybrid_clock.cc:648] HybridClock initialized: now 1790395385954013 us; error 51 us; skew 500 ppm
Sep 26 04:03:05 dist-test-slave-j8c1 krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:05.957039 3302 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:05.957607 3302 webserver.cc:460] Webserver started at http://127.0.200.129:33627/ using document root <none> and password file <none>
I20260926 04:03:05.957861 3302 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:05.957931 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:05.958078 3302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:05.959259 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "fa7f1973fd984ce6b8fb1879e0557541"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "3e8b25e9ac8c7377a97be3cda090e7d0"
server_key_iv: "088eee6f46db62989689f18de76f50f2"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.959697 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "fa7f1973fd984ce6b8fb1879e0557541"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "3e8b25e9ac8c7377a97be3cda090e7d0"
server_key_iv: "088eee6f46db62989689f18de76f50f2"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.961162 3302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:05.962091 3317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:05.962281 3302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:05.962388 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "fa7f1973fd984ce6b8fb1879e0557541"
format_stamp: "Formatted at 2026-09-26 04:03:05 on dist-test-slave-j8c1"
server_key: "3e8b25e9ac8c7377a97be3cda090e7d0"
server_key_iv: "088eee6f46db62989689f18de76f50f2"
server_key_version: "encryptionkey@0"
I20260926 04:03:05.962471 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:05.981840 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:05.984717 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:05.984928 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:05.985203 3302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:05.985567 3302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:05.985623 3302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:05.985666 3302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:05.985695 3302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:05.991797 3302 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:44985
I20260926 04:03:05.991842 3430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:44985 every 8 connection(s)
I20260926 04:03:05.992213 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:05 dist-test-slave-j8c1 krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:06.000269 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3302
I20260926 04:03:06.000432 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:06.000638 802 external_mini_cluster.cc:1428] Setting key 14a10fc386a6595d8351c9e78abacdfa
I20260926 04:03:06.003436 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:05.993027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:38565 (local address 127.0.200.190:43063)
0926 04:03:05.993073 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:05.993075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:05.993372 (+ 297us) server_negotiation.cc:407] Connection header received
0926 04:03:05.994396 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:05.994397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:05.994418 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:05.994486 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:05.995180 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.995775 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.997300 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.997521 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.997590 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:06.000154 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:06.000156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:06.000157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:06.000167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:06.001890 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.002341 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.002343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.002343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.002412 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.002733 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.002734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.002734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.002866 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:06.002958 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:06.003243 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 04:03:06.003341 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:03:06.003827 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:05.992962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43063 (local address 127.0.200.129:38565)
0926 04:03:05.993212 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:05.993243 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:05.994276 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:05.994608 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:05.994612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:05.994888 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:05.995088 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:05.995092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.995961 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.995963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:05.997180 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:05.997185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:05.997633 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:05.997634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:05.997757 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:05.998439 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:05.998449 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:06.000006 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:06.001984 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.001994 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.001997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.002251 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.002495 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.002496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.002496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.002662 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.002936 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:06.002943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:06.003101 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 04:03:06.003208 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 04:03:06.003390 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260926 04:03:06.004295 3431 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43063
I20260926 04:03:06.004393 3431 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:06.004604 3431 heartbeater.cc:511] Master 127.0.200.190:43063 requested a full tablet report, sending...
I20260926 04:03:06.005126 3240 ts_manager.cc:194] Registered new tserver with Master: fa7f1973fd984ce6b8fb1879e0557541 (127.0.200.129:44985)
I20260926 04:03:06.005789 3240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:38565
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:06.030682 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:06.117534 3436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:06.117765 3436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:06.117792 3436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:06.119185 3436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:06.119236 3436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:06.119266 3436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:06.120893 3436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:06.121184 3436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:06.121439 3436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:06.126760 3442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:06.126760 3441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:06.127096 3444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:06.127573 3436 server_base.cc:1034] running on GCE node
I20260926 04:03:06.127789 3436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:06.128000 3436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:06.129155 3436 hybrid_clock.cc:648] HybridClock initialized: now 1790395386129142 us; error 30 us; skew 500 ppm
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:06.131965 3436 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:06.132427 3436 webserver.cc:460] Webserver started at http://127.0.200.130:33251/ using document root <none> and password file <none>
I20260926 04:03:06.132704 3436 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:06.132779 3436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:06.132932 3436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:06.134013 3436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "1283ce73a61c428da5b0f5fb20d78a2e"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "fbe5c9de45c900a203aab2034cedd56a"
server_key_iv: "6a8eecc0d35f33d9bfcc607c48c7b0ce"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.134384 3436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "1283ce73a61c428da5b0f5fb20d78a2e"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "fbe5c9de45c900a203aab2034cedd56a"
server_key_iv: "6a8eecc0d35f33d9bfcc607c48c7b0ce"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.136020 3436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:06.136821 3451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.137068 3436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:06.137158 3436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "1283ce73a61c428da5b0f5fb20d78a2e"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "fbe5c9de45c900a203aab2034cedd56a"
server_key_iv: "6a8eecc0d35f33d9bfcc607c48c7b0ce"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.137236 3436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:06.156626 3436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:06.159296 3436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:06.159430 3436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:06.159621 3436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:06.159986 3436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:06.160045 3436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.160089 3436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:06.160116 3436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.166193 3436 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:40607
I20260926 04:03:06.166266 3564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:40607 every 8 connection(s)
I20260926 04:03:06.166704 3436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:06.176613 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:06.167499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:47405 (local address 127.0.200.190:43063)
0926 04:03:06.167568 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:06.167570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:06.167785 (+ 215us) server_negotiation.cc:407] Connection header received
0926 04:03:06.168816 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:06.168817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:06.168830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:06.168877 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:06.169539 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.170045 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.171137 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.171352 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.171442 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:06.173756 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:06.173758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:06.173758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:06.173766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:06.175244 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.175686 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.175687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.175687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.175753 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.176011 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.176011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.176012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.176135 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:06.176190 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:06.176466 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 04:03:06.176544 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:03:06.176613 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3436
I20260926 04:03:06.176721 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:06.176957 802 external_mini_cluster.cc:1428] Setting key d1cfe3f46fe32a882980982966c7ff40
I20260926 04:03:06.176959 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:06.167481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43063 (local address 127.0.200.130:47405)
0926 04:03:06.167666 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:06.167703 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:06.168687 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:06.168971 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:06.168975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:06.169244 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:06.169444 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:06.169448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.170141 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.170142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:06.171034 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:06.171037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.171450 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.171451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:06.171582 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:06.172114 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:06.172123 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:06.173623 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:06.175322 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.175329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.175331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.175584 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.175826 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.175826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.175827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.175946 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.176200 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:06.176204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:06.176324 (+ 120us) client_negotiation.cc:769] Sending connection context
0926 04:03:06.176445 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 04:03:06.176628 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260926 04:03:06.177570 3565 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43063
I20260926 04:03:06.177675 3565 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:06.177929 3565 heartbeater.cc:511] Master 127.0.200.190:43063 requested a full tablet report, sending...
I20260926 04:03:06.178364 3239 ts_manager.cc:194] Registered new tserver with Master: 1283ce73a61c428da5b0f5fb20d78a2e (127.0.200.130:40607)
I20260926 04:03:06.178812 3239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:47405
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:06.207147 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:06.293907 3570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:06.294130 3570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:06.294155 3570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:06.295389 3570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:06.295434 3570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:06.295464 3570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:06.296993 3570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:06.297184 3570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:06.297369 3570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:06.302816 3578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:06.302805 3575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:06.302889 3570 server_base.cc:1034] running on GCE node
W20260926 04:03:06.302830 3576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:06.303243 3570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:06.303488 3570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:06.304644 3570 hybrid_clock.cc:648] HybridClock initialized: now 1790395386304623 us; error 47 us; skew 500 ppm
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:06.307379 3570 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:06.307770 3570 webserver.cc:460] Webserver started at http://127.0.200.131:44317/ using document root <none> and password file <none>
I20260926 04:03:06.308001 3570 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:06.308070 3570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:06.308225 3570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:06.309284 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "69c2bad16f2240ae947eca85e7f2fc96"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "79bec4fecad5f7922d07d5e4d6682481"
server_key_iv: "197607794948fce4520c5264111ccc02"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.309657 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "69c2bad16f2240ae947eca85e7f2fc96"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "79bec4fecad5f7922d07d5e4d6682481"
server_key_iv: "197607794948fce4520c5264111ccc02"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.311164 3570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:06.312131 3585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.312342 3570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:06.312448 3570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "69c2bad16f2240ae947eca85e7f2fc96"
format_stamp: "Formatted at 2026-09-26 04:03:06 on dist-test-slave-j8c1"
server_key: "79bec4fecad5f7922d07d5e4d6682481"
server_key_iv: "197607794948fce4520c5264111ccc02"
server_key_version: "encryptionkey@0"
I20260926 04:03:06.312548 3570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:06.324620 3570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:06.327060 3570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:06.327239 3570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:06.327488 3570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:06.327800 3570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:06.327852 3570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.327896 3570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:06.327927 3570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:06.333667 3570 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:43983
I20260926 04:03:06.333745 3698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:43983 every 8 connection(s)
I20260926 04:03:06.334081 3570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:06.342648 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3570
I20260926 04:03:06.342758 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:06.342927 802 external_mini_cluster.cc:1428] Setting key 5394eed4e0ffddb8072dffcefc420eab
I20260926 04:03:06.344781 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:06.334977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:43127 (local address 127.0.200.190:43063)
0926 04:03:06.335069 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:06.335072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:06.335225 (+ 153us) server_negotiation.cc:407] Connection header received
0926 04:03:06.336171 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:06.336173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:06.336184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:06.336238 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:06.336866 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.337383 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.338401 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.338665 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.338721 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:06.341201 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:06.341203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:06.341203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:06.341213 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:06.343199 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.343730 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.343731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.343732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.343807 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:06.344100 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:06.344101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:06.344101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:06.344233 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:06.344311 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:06.344598 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 04:03:06.344681 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 04:03:06.345006 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:06.334934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43063 (local address 127.0.200.131:43127)
0926 04:03:06.335097 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:06.335133 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:06.336057 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:06.336297 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:06.336301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:06.336561 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:06.336754 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:06.336758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.337484 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.337486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:06.338310 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:06.338312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:06.338738 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:06.338738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:06.338843 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:06.339424 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:06.339433 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:06.340868 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:06.343292 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.343301 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.343304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.343635 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.343876 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:06.343877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:06.343877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:06.344028 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:06.344303 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:06.344309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:06.344461 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 04:03:06.344576 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 04:03:06.344749 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260926 04:03:06.345546 3699 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43063
I20260926 04:03:06.345634 3699 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:06.345821 3699 heartbeater.cc:511] Master 127.0.200.190:43063 requested a full tablet report, sending...
I20260926 04:03:06.346287 3239 ts_manager.cc:194] Registered new tserver with Master: 69c2bad16f2240ae947eca85e7f2fc96 (127.0.200.131:43983)
I20260926 04:03:06.346802 3239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:43127
I20260926 04:03:06.355193 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 04:03:06.366858 3295 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:06.365806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45450 (local address 127.0.200.190:43063)
0926 04:03:06.365862 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:06.365865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:06.365880 (+ 15us) server_negotiation.cc:407] Connection header received
0926 04:03:06.366049 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:06.366051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:06.366065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:06.366138 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:06.366777 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45450: BlockingRecv error: recv got EOF from 127.0.0.1:45450 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:03:06.367865 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3302
I20260926 04:03:06.372826 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3436
I20260926 04:03:06.377758 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3570
I20260926 04:03:06.382115 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3208
2026-09-26T04:03:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1923 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3711](info): set up 2 sockets
Sep 26 04:03:06 dist-test-slave-j8c1 krb5kdc[3711](info): commencing operation
krb5kdc: starting...
W20260926 04:03:07.442723 802 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:07.566172 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37295
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:07.652305 3724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:07.652604 3724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:07.652647 3724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:07.654072 3724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:07.654157 3724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:07.654191 3724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:07.654222 3724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:07.654258 3724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:07.656838 3724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:07.657188 3724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:07.657442 3724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:07.662736 3730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:07.662696 3729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:07.662743 3732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:07.664347 3724 server_base.cc:1034] running on GCE node
I20260926 04:03:07.664645 3724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:07.664956 3724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:07.666138 3724 hybrid_clock.cc:648] HybridClock initialized: now 1790395387666130 us; error 61 us; skew 500 ppm
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:07.669062 3724 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:07.669602 3724 webserver.cc:460] Webserver started at http://127.0.200.190:42327/ using document root <none> and password file <none>
I20260926 04:03:07.669801 3724 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:07.669842 3724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:07.669940 3724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:07.671235 3724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "3cd9dfc064b8432cae79833edc13f59f"
format_stamp: "Formatted at 2026-09-26 04:03:07 on dist-test-slave-j8c1"
server_key: "aecbf7a15585baba3789b62a0a53d5c6"
server_key_iv: "586115cc737bc4bcc16a6f8837efac1d"
server_key_version: "encryptionkey@0"
I20260926 04:03:07.671546 3724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "3cd9dfc064b8432cae79833edc13f59f"
format_stamp: "Formatted at 2026-09-26 04:03:07 on dist-test-slave-j8c1"
server_key: "aecbf7a15585baba3789b62a0a53d5c6"
server_key_iv: "586115cc737bc4bcc16a6f8837efac1d"
server_key_version: "encryptionkey@0"
I20260926 04:03:07.672964 3724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:03:07.674082 3739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:07.674335 3724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:07.674443 3724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "3cd9dfc064b8432cae79833edc13f59f"
format_stamp: "Formatted at 2026-09-26 04:03:07 on dist-test-slave-j8c1"
server_key: "aecbf7a15585baba3789b62a0a53d5c6"
server_key_iv: "586115cc737bc4bcc16a6f8837efac1d"
server_key_version: "encryptionkey@0"
I20260926 04:03:07.674525 3724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:07.701750 3724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:07.704437 3724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:07.704610 3724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:07.708722 3724 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:37295
I20260926 04:03:07.709074 3791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:37295 every 8 connection(s)
I20260926 04:03:07.709126 3724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:07.709879 3792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:07.712337 3792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f: Bootstrap starting.
I20260926 04:03:07.712958 3792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:07.713275 3792 log.cc:824] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:07.714161 3792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f: No bootstrap required, opened a new log
I20260926 04:03:07.714181 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3724
I20260926 04:03:07.714294 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:07.714478 802 external_mini_cluster.cc:1428] Setting key 84e1dd8b7faf90901da39c002079ffec
I20260926 04:03:07.716776 3792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } }
I20260926 04:03:07.716964 3792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:07.717010 3792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd9dfc064b8432cae79833edc13f59f, State: Initialized, Role: FOLLOWER
I20260926 04:03:07.717181 3792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } }
I20260926 04:03:07.717273 3792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:07.717319 3792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:07.717374 3792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:07.717972 3792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } }
I20260926 04:03:07.718096 3792 leader_election.cc:302] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cd9dfc064b8432cae79833edc13f59f; no voters:
I20260926 04:03:07.718321 3792 leader_election.cc:288] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:07.718353 3797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:07.718456 3797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [term 1 LEADER]: Becoming Leader. State: Replica: 3cd9dfc064b8432cae79833edc13f59f, State: Running, Role: LEADER
I20260926 04:03:07.718624 3792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:07.718590 3797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } }
I20260926 04:03:07.718968 3799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cd9dfc064b8432cae79833edc13f59f. Latest consensus state: current_term: 1 leader_uuid: "3cd9dfc064b8432cae79833edc13f59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } } }
I20260926 04:03:07.719095 3799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:07.719293 3798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cd9dfc064b8432cae79833edc13f59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd9dfc064b8432cae79833edc13f59f" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37295 } } }
I20260926 04:03:07.719357 3798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:07.719419 3803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:07.720054 3803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:07.723841 3803 catalog_manager.cc:1347] Generated new cluster ID: 34d655848f3945c2ae2a2cd4e6046a52
I20260926 04:03:07.723899 3803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:07.729130 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:07.715106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.0.200.190:37295)
0926 04:03:07.715281 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:07.715289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:07.715317 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:03:07.716779 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:07.716790 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:07.716858 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:07.717063 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:07.717943 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:07.719569 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:07.720834 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:07.721311 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:07.721440 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:07.724492 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:07.724496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:07.724496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:07.724510 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:07.726701 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:07.727236 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:07.727239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:07.727240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:07.727376 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:07.727631 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:07.727632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:07.727633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:07.728024 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:07.728232 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:07.728371 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 04:03:07.728645 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260926 04:03:07.730152 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:03:07.733490 3803 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:07.734316 3803 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:07.739586 3803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cd9dfc064b8432cae79833edc13f59f: Generated new TSK 0
I20260926 04:03:07.739877 3803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:07.748095 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:07.743743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.0.200.190:37295)
0926 04:03:07.743815 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:07.743817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:07.743828 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:07.743856 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:07.743857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:07.743868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:07.743938 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:07.744373 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:07.745187 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:07.746072 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:07.746421 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:07.746500 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:07.747564 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:07.747566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:07.747566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:07.747827 (+ 261us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:07.747848 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:07.747855 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:07.747920 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:07.747972 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 04:03:07.748002 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:03:07.751091 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3724
2026-09-26T04:03:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1366 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3825](info): set up 2 sockets
Sep 26 04:03:07 dist-test-slave-j8c1 krb5kdc[3825](info): commencing operation
krb5kdc: starting...
W20260926 04:03:08.838905 802 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:08 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:08.973732 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:41545
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:09.057108 3838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:09.057353 3838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:09.057379 3838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:09.058771 3838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:09.058837 3838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:09.058852 3838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:09.058866 3838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:09.058878 3838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:09.061070 3838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:09.061355 3838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:09.061616 3838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:09.066970 3844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.066978 3843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:09.067142 3838 server_base.cc:1034] running on GCE node
W20260926 04:03:09.066970 3846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:09.067536 3838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:09.067865 3838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:09.069041 3838 hybrid_clock.cc:648] HybridClock initialized: now 1790395389069026 us; error 29 us; skew 500 ppm
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:09.078971 3838 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:09.079550 3838 webserver.cc:460] Webserver started at http://127.0.200.190:43983/ using document root <none> and password file <none>
I20260926 04:03:09.079815 3838 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:09.079897 3838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:09.080065 3838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:09.081430 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "79ba76ac151b468e94f074f4512eda26"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "b743bf5e1c4d820af0da59a6713d81b5"
server_key_iv: "e3886a1ea9bac67c0f03b676c85d51cd"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.081964 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "79ba76ac151b468e94f074f4512eda26"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "b743bf5e1c4d820af0da59a6713d81b5"
server_key_iv: "e3886a1ea9bac67c0f03b676c85d51cd"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.084076 3838 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260926 04:03:09.085044 3853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.085328 3838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:09.085521 3838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "79ba76ac151b468e94f074f4512eda26"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "b743bf5e1c4d820af0da59a6713d81b5"
server_key_iv: "e3886a1ea9bac67c0f03b676c85d51cd"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.085680 3838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:09.103329 3838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:09.106001 3838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:09.106202 3838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:09.110800 3838 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:41545
I20260926 04:03:09.110867 3905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:41545 every 8 connection(s)
I20260926 04:03:09.111267 3838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:09.112555 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:09.115375 3906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: Bootstrap starting.
I20260926 04:03:09.116312 3906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:09.117080 3906 log.cc:824] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:09.117991 3906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: No bootstrap required, opened a new log
I20260926 04:03:09.119777 3906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } }
I20260926 04:03:09.119966 3906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:09.119988 3906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ba76ac151b468e94f074f4512eda26, State: Initialized, Role: FOLLOWER
I20260926 04:03:09.120188 3906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } }
I20260926 04:03:09.120286 3906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:09.120252 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3838
I20260926 04:03:09.120312 3906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:09.120399 3906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:09.120527 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:09.120802 802 external_mini_cluster.cc:1428] Setting key 9d6995743667a820daf0738c5b17ab9f
I20260926 04:03:09.121564 3906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } }
I20260926 04:03:09.121714 3906 leader_election.cc:302] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ba76ac151b468e94f074f4512eda26; no voters:
I20260926 04:03:09.121904 3906 leader_election.cc:288] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:09.121951 3911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:09.122092 3911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [term 1 LEADER]: Becoming Leader. State: Replica: 79ba76ac151b468e94f074f4512eda26, State: Running, Role: LEADER
I20260926 04:03:09.122184 3911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } }
I20260926 04:03:09.122187 3906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:09.123746 3912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ba76ac151b468e94f074f4512eda26. Latest consensus state: current_term: 1 leader_uuid: "79ba76ac151b468e94f074f4512eda26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } } }
I20260926 04:03:09.123914 3912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [sys.catalog]: This master's current role is: LEADER
W20260926 04:03:09.124313 3925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 04:03:09.124287 3913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ba76ac151b468e94f074f4512eda26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ba76ac151b468e94f074f4512eda26" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 41545 } } }
W20260926 04:03:09.124358 3925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:09.124367 3913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:09.124409 3927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:09.124581 3927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:09.126216 3927 catalog_manager.cc:1347] Generated new cluster ID: 0f807ca73ab74495aa03c431fedf525a
I20260926 04:03:09.126281 3927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:09.133014 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.121378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.0.200.190:41545)
0926 04:03:09.121574 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:09.121583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:09.121612 (+ 29us) server_negotiation.cc:407] Connection header received
0926 04:03:09.121761 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:09.121769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:09.121849 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:09.122026 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:09.122997 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.124667 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.125461 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.126097 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.126249 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.129169 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:09.129174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:09.129175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:09.129191 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:09.131228 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.131658 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.131662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.131663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.131773 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.132014 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.132015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.132015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.132300 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:09.132482 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:09.132592 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 04:03:09.132761 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260926 04:03:09.145138 3927 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:09.146000 3927 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:09.150308 3927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ba76ac151b468e94f074f4512eda26: Generated new TSK 0
I20260926 04:03:09.150580 3927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:09.183002 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:09.277292 3933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:09.277522 3933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:09.277550 3933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:09.278908 3933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:09.278964 3933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:09.278995 3933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:09.281025 3933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:09.281358 3933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:09.281625 3933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:09.287303 3941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.287310 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.287310 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:09.287833 3933 server_base.cc:1034] running on GCE node
I20260926 04:03:09.288075 3933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:09.288308 3933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:09.289516 3933 hybrid_clock.cc:648] HybridClock initialized: now 1790395389289503 us; error 41 us; skew 500 ppm
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:09.292623 3933 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:09.293197 3933 webserver.cc:460] Webserver started at http://127.0.200.129:36029/ using document root <none> and password file <none>
I20260926 04:03:09.293490 3933 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:09.293541 3933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:09.293687 3933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:09.294921 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "363c4c9f145f4d5fa4d3494e738b8796"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "8e3b8e44ea1ccfb371e583b4331f66ea"
server_key_iv: "15320c3ab57251b7257fd5a107124cfa"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.295290 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "363c4c9f145f4d5fa4d3494e738b8796"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "8e3b8e44ea1ccfb371e583b4331f66ea"
server_key_iv: "15320c3ab57251b7257fd5a107124cfa"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.296758 3933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:09.297716 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.298014 3933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:09.298138 3933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "363c4c9f145f4d5fa4d3494e738b8796"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "8e3b8e44ea1ccfb371e583b4331f66ea"
server_key_iv: "15320c3ab57251b7257fd5a107124cfa"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.298265 3933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:09.315029 3933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:09.317539 3933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:09.317723 3933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:09.317968 3933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:09.318300 3933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:09.318353 3933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.318394 3933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:09.318421 3933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.324942 3933 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:46541
I20260926 04:03:09.324985 4061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:46541 every 8 connection(s)
I20260926 04:03:09.325344 3933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:09.330276 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 3933
I20260926 04:03:09.330415 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:09.330648 802 external_mini_cluster.cc:1428] Setting key a411a46ec036e5995bcfa99e19354cc0
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:09.337452 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.326179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:46153 (local address 127.0.200.190:41545)
0926 04:03:09.326227 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:09.326229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:09.326447 (+ 218us) server_negotiation.cc:407] Connection header received
0926 04:03:09.327072 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:09.327074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:09.327090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:09.327176 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:09.328195 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.328727 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.329832 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.330120 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.330210 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.333398 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:09.333400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:09.333401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:09.333411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:09.335430 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.336015 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.336017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.336017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.336090 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.336375 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.336376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.336376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.336640 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:09.336793 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:09.337209 (+ 416us) server_negotiation.cc:299] Negotiation successful
0926 04:03:09.337334 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:03:09.337831 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.326104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:41545 (local address 127.0.200.129:46153)
0926 04:03:09.326276 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:09.326352 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:09.326920 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:09.327242 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:09.327246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:09.327869 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:09.328073 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.328078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.328830 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.328831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.329698 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.329701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.330311 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.330313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.330751 (+ 438us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.331713 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:09.331727 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:09.333278 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:09.335592 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.335602 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.335605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.335896 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.336167 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.336169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.336169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.336294 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.336801 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:09.336814 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:09.337032 (+ 218us) client_negotiation.cc:769] Sending connection context
0926 04:03:09.337166 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 04:03:09.337387 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260926 04:03:09.338446 4062 heartbeater.cc:347] Connected to a master server at 127.0.200.190:41545
I20260926 04:03:09.338596 4062 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:09.338853 4062 heartbeater.cc:511] Master 127.0.200.190:41545 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:03:09.339486 3870 ts_manager.cc:194] Registered new tserver with Master: 363c4c9f145f4d5fa4d3494e738b8796 (127.0.200.129:46541)
I20260926 04:03:09.340116 3870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:46153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:09.363490 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:09.453279 4067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:09.453521 4067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:09.453545 4067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:09.454751 4067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:09.454797 4067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:09.454825 4067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:09.456593 4067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:09.456841 4067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:09.457039 4067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:09.462417 4072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.462443 4075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.462558 4073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:09.462738 4067 server_base.cc:1034] running on GCE node
I20260926 04:03:09.463109 4067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:09.463373 4067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:09.464533 4067 hybrid_clock.cc:648] HybridClock initialized: now 1790395389464508 us; error 46 us; skew 500 ppm
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:09.467276 4067 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:09.467690 4067 webserver.cc:460] Webserver started at http://127.0.200.130:34849/ using document root <none> and password file <none>
I20260926 04:03:09.468039 4067 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:09.468120 4067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:09.468303 4067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:09.469446 4067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "1da05d07bf9f4d2096ab581f04d1d4ec"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "74489af936e2dbd4047a29bd93cc5cd0"
server_key_iv: "dd336e9482e82260f939cd159cef2900"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.469834 4067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "1da05d07bf9f4d2096ab581f04d1d4ec"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "74489af936e2dbd4047a29bd93cc5cd0"
server_key_iv: "dd336e9482e82260f939cd159cef2900"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.471518 4067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:09.472734 4082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.473088 4067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:09.473277 4067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "1da05d07bf9f4d2096ab581f04d1d4ec"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "74489af936e2dbd4047a29bd93cc5cd0"
server_key_iv: "dd336e9482e82260f939cd159cef2900"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.473385 4067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:09.500054 4067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:09.502737 4067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:09.502948 4067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:09.503203 4067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:09.503542 4067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:09.503597 4067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.503639 4067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:09.503667 4067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.509847 4067 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:45421
I20260926 04:03:09.509888 4195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:45421 every 8 connection(s)
I20260926 04:03:09.510233 4067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:09.510680 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4067
I20260926 04:03:09.510798 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:09.510989 802 external_mini_cluster.cc:1428] Setting key 5e62b0d31cc8f1fe2e500397b9e676fa
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:03:09.522500 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.511201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:35441 (local address 127.0.200.190:41545)
0926 04:03:09.511272 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:09.511275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:09.511555 (+ 280us) server_negotiation.cc:407] Connection header received
0926 04:03:09.512227 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:09.512228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:09.512238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:09.512314 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:09.513696 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.514496 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.516212 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.516415 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.516568 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.519372 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:09.519374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:09.519375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:09.519390 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:09.520888 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.521384 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.521385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.521385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.521447 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.521728 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.521730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.521730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.521859 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:09.521935 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:09.522310 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 04:03:09.522399 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 04:03:09.522792 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.511105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:41545 (local address 127.0.200.130:35441)
0926 04:03:09.511411 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:09.511460 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:09.512118 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:09.512368 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:09.512372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:09.513252 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:09.513586 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.513600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.514636 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.514637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.516115 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.516118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.516576 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.516577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.516738 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.517355 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:09.517376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:09.519252 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:09.521003 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.521012 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.521015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.521301 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.521522 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.521523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.521523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.521645 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.521936 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:09.521943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:09.522149 (+ 206us) client_negotiation.cc:769] Sending connection context
0926 04:03:09.522279 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 04:03:09.522457 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":218,"threads_started":1}
I20260926 04:03:09.523265 4196 heartbeater.cc:347] Connected to a master server at 127.0.200.190:41545
I20260926 04:03:09.523368 4196 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:09.523521 4196 heartbeater.cc:511] Master 127.0.200.190:41545 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
I20260926 04:03:09.524021 3870 ts_manager.cc:194] Registered new tserver with Master: 1da05d07bf9f4d2096ab581f04d1d4ec (127.0.200.130:45421)
I20260926 04:03:09.524432 3870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:35441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:09.543310 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:09.634057 4201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:09.634308 4201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:09.634337 4201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:09.635739 4201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:09.635800 4201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:09.635831 4201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:09.637408 4201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:09.637672 4201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:09.637928 4201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:09.643249 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.643278 4209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:09.643390 4206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:09.643350 4201 server_base.cc:1034] running on GCE node
I20260926 04:03:09.643751 4201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:09.644016 4201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:09.645217 4201 hybrid_clock.cc:648] HybridClock initialized: now 1790395389645213 us; error 38 us; skew 500 ppm
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:09.648283 4201 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:09.648917 4201 webserver.cc:460] Webserver started at http://127.0.200.131:35099/ using document root <none> and password file <none>
I20260926 04:03:09.649266 4201 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:09.649362 4201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:09.649550 4201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:09.650849 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "5f550e65da154bf8ae723bb953516035"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "e6cdcbe66e7da34e246abc07f9b73eee"
server_key_iv: "e0632ecfe2efe5514754dd039edffd9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.651237 4201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "5f550e65da154bf8ae723bb953516035"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "e6cdcbe66e7da34e246abc07f9b73eee"
server_key_iv: "e0632ecfe2efe5514754dd039edffd9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.652679 4201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:09.653548 4216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.653772 4201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:09.653898 4201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "5f550e65da154bf8ae723bb953516035"
format_stamp: "Formatted at 2026-09-26 04:03:09 on dist-test-slave-j8c1"
server_key: "e6cdcbe66e7da34e246abc07f9b73eee"
server_key_iv: "e0632ecfe2efe5514754dd039edffd9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:09.653978 4201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:09.664343 4201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:09.666774 4201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:09.666951 4201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:09.667196 4201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:09.667522 4201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:09.667578 4201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.667620 4201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:09.667651 4201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:09.673589 4201 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:40135
I20260926 04:03:09.673627 4329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:40135 every 8 connection(s)
I20260926 04:03:09.673982 4201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:09.679462 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4201
I20260926 04:03:09.679590 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:09.679829 802 external_mini_cluster.cc:1428] Setting key cce7e1cc445789640e40962dd39d14c4
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:09.685029 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.674889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:44491 (local address 127.0.200.190:41545)
0926 04:03:09.674966 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:09.674968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:09.675227 (+ 259us) server_negotiation.cc:407] Connection header received
0926 04:03:09.675731 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:09.675732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:09.675742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:09.675809 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:09.676927 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.677600 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.678702 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.678909 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.678986 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.681706 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:09.681708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:09.681708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:09.681718 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:09.683413 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.683897 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.683898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.683899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.683961 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.684233 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.684234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.684234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.684363 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:09.684439 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:09.684837 (+ 398us) server_negotiation.cc:299] Negotiation successful
0926 04:03:09.684927 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:03:09.685282 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.674893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:41545 (local address 127.0.200.131:44491)
0926 04:03:09.675096 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:09.675133 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:09.675631 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:09.675900 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:09.675904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:09.676597 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:09.676802 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.676814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.677709 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.677710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.678615 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:09.678617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.679017 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.679018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:09.679133 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.679752 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:09.679764 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:09.681589 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:09.683536 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.683546 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.683549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.683802 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.684029 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:09.684030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:09.684030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:09.684158 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:09.684430 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:09.684436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:09.684656 (+ 220us) client_negotiation.cc:769] Sending connection context
0926 04:03:09.684814 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 04:03:09.685005 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260926 04:03:09.685854 4330 heartbeater.cc:347] Connected to a master server at 127.0.200.190:41545
I20260926 04:03:09.685951 4330 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:09.686151 4330 heartbeater.cc:511] Master 127.0.200.190:41545 requested a full tablet report, sending...
I20260926 04:03:09.686709 3870 ts_manager.cc:194] Registered new tserver with Master: 5f550e65da154bf8ae723bb953516035 (127.0.200.131:40135)
I20260926 04:03:09.687083 3870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:44491
I20260926 04:03:09.692219 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[3825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395389, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:09.717705 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:09.709644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.0.200.190:41545)
0926 04:03:09.709765 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:09.709767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:09.709964 (+ 197us) server_negotiation.cc:407] Connection header received
0926 04:03:09.709996 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:09.709997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:09.710008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:09.710076 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:09.710788 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.711304 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.712138 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:09.712355 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:09.712424 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:09.714950 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:09.714951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:09.714951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:09.714959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:09.716402 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.716813 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.716814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.716814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.716872 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:09.717147 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:09.717148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:09.717148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:09.717318 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:09.717407 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:09.717538 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 04:03:09.717604 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 04:03:09.718032 3870 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55240
I20260926 04:03:09.718528 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3933
I20260926 04:03:09.723177 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4067
I20260926 04:03:09.727910 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4201
I20260926 04:03:09.732177 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 3838
2026-09-26T04:03:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1982 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[4342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[4342](info): set up 2 sockets
Sep 26 04:03:09 dist-test-slave-j8c1 krb5kdc[4342](info): commencing operation
krb5kdc: starting...
W20260926 04:03:10.802809 802 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:10 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:10.935178 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37675
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:11.018666 4355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:11.018896 4355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:11.018927 4355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:11.020283 4355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:11.020337 4355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:11.020352 4355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:11.020365 4355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:11.020377 4355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:11.021960 4355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:11.022243 4355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:11.022504 4355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:11.027727 4360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:11.027714 4363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:11.027714 4361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.027977 4355 server_base.cc:1034] running on GCE node
I20260926 04:03:11.028285 4355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:11.028631 4355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:11.029824 4355 hybrid_clock.cc:648] HybridClock initialized: now 1790395391029808 us; error 35 us; skew 500 ppm
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:11.032721 4355 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:11.033295 4355 webserver.cc:460] Webserver started at http://127.0.200.190:44265/ using document root <none> and password file <none>
I20260926 04:03:11.033556 4355 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:11.033627 4355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:11.033779 4355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:11.034996 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "b187b12b45414cb687eb726741ddffd5"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "39d2f3c471f61487629ba87b711ad770"
server_key_iv: "54e48bdeacf0a71a49fba180b0fa5fde"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.035377 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "b187b12b45414cb687eb726741ddffd5"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "39d2f3c471f61487629ba87b711ad770"
server_key_iv: "54e48bdeacf0a71a49fba180b0fa5fde"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.036836 4355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:11.037856 4370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.038125 4355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:11.038245 4355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "b187b12b45414cb687eb726741ddffd5"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "39d2f3c471f61487629ba87b711ad770"
server_key_iv: "54e48bdeacf0a71a49fba180b0fa5fde"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.038331 4355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:11.065148 4355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:11.067718 4355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:11.067907 4355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:11.072636 4355 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:37675
I20260926 04:03:11.072675 4422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:37675 every 8 connection(s)
I20260926 04:03:11.073043 4355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:11.074028 4423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:11.076534 4423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: Bootstrap starting.
I20260926 04:03:11.077113 4423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:11.077395 4423 log.cc:824] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:11.078138 4423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: No bootstrap required, opened a new log
I20260926 04:03:11.079844 4423 raft_consensus.cc:348] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } }
I20260926 04:03:11.080017 4423 raft_consensus.cc:374] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:11.080044 4423 raft_consensus.cc:729] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b187b12b45414cb687eb726741ddffd5, State: Initialized, Role: FOLLOWER
I20260926 04:03:11.080204 4423 consensus_queue.cc:260] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } }
I20260926 04:03:11.080281 4423 raft_consensus.cc:388] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:11.080303 4423 raft_consensus.cc:482] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:11.080367 4423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:11.080722 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4355
I20260926 04:03:11.080852 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:11.081063 802 external_mini_cluster.cc:1428] Setting key 13f8d9ee5bdc3ead48b182515b30fd5a
I20260926 04:03:11.081048 4423 raft_consensus.cc:504] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } }
I20260926 04:03:11.081177 4423 leader_election.cc:302] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b187b12b45414cb687eb726741ddffd5; no voters:
I20260926 04:03:11.081391 4423 leader_election.cc:288] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:11.081468 4426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:11.081699 4423 sys_catalog.cc:564] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:11.082031 4426 raft_consensus.cc:686] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [term 1 LEADER]: Becoming Leader. State: Replica: b187b12b45414cb687eb726741ddffd5, State: Running, Role: LEADER
I20260926 04:03:11.082299 4426 consensus_queue.cc:237] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } }
I20260926 04:03:11.082736 4427 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b187b12b45414cb687eb726741ddffd5. Latest consensus state: current_term: 1 leader_uuid: "b187b12b45414cb687eb726741ddffd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } } }
I20260926 04:03:11.082983 4427 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:11.082913 4430 sys_catalog.cc:455] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b187b12b45414cb687eb726741ddffd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b187b12b45414cb687eb726741ddffd5" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37675 } } }
I20260926 04:03:11.083155 4430 sys_catalog.cc:458] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5 [sys.catalog]: This master's current role is: LEADER
W20260926 04:03:11.084043 4442 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:03:11.084112 4442 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:11.084161 4443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:11.084342 4443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:11.086473 4443 catalog_manager.cc:1347] Generated new cluster ID: e96c32235dde47c980e7ee5fb73f9f55
I20260926 04:03:11.086583 4443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:11.093591 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.081775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.0.200.190:37675)
0926 04:03:11.082428 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.082442 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.082487 (+ 45us) server_negotiation.cc:407] Connection header received
0926 04:03:11.082673 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.082681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.082755 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.083515 (+ 760us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.083838 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.085720 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.086969 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.087453 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.087586 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.090054 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.090059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.090059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.090074 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.091850 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.092253 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.092256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.092257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.092354 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.092569 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.092570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.092570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.092863 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.093059 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.093131 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.093397 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":89,"threads_started":1}
I20260926 04:03:11.094681 4443 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:11.095238 4443 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:11.099432 4443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b187b12b45414cb687eb726741ddffd5: Generated new TSK 0
I20260926 04:03:11.099831 4443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:11.139541 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:11.243665 4449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:11.243903 4449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:11.243930 4449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:11.246065 4449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:11.246165 4449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:11.246205 4449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:11.249383 4449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:11.249764 4449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:11.250041 4449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:11.256196 4455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:11.256173 4454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.256477 4449 server_base.cc:1034] running on GCE node
W20260926 04:03:11.256198 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.256827 4449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:11.257143 4449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:11.258345 4449 hybrid_clock.cc:648] HybridClock initialized: now 1790395391258323 us; error 40 us; skew 500 ppm
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:11.261873 4449 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:11.262507 4449 webserver.cc:460] Webserver started at http://127.0.200.129:34123/ using document root <none> and password file <none>
I20260926 04:03:11.262820 4449 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:11.262897 4449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:11.263072 4449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:11.264709 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "0f37a53d5caf44b9902a8306601937db"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d28728ed488968d0c638e243af466242"
server_key_iv: "65356ace08a956fb842df436ae0570df"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.265412 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "0f37a53d5caf44b9902a8306601937db"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d28728ed488968d0c638e243af466242"
server_key_iv: "65356ace08a956fb842df436ae0570df"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.267424 4449 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260926 04:03:11.268625 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.268944 4449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:11.269088 4449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "0f37a53d5caf44b9902a8306601937db"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d28728ed488968d0c638e243af466242"
server_key_iv: "65356ace08a956fb842df436ae0570df"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.269156 4449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:11.287067 4449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:11.289362 4449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:11.289529 4449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:11.289748 4449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:11.290086 4449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:11.290122 4449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.290145 4449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:11.290158 4449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.296911 4449 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:34003
I20260926 04:03:11.296959 4577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:34003 every 8 connection(s)
I20260926 04:03:11.297293 4449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:11.306671 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4449
I20260926 04:03:11.306770 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:11.306991 802 external_mini_cluster.cc:1428] Setting key f8ad02c762a342faec12c869856c4868
I20260926 04:03:11.308641 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.298306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:42571 (local address 127.0.200.190:37675)
0926 04:03:11.298368 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.298370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.298417 (+ 47us) server_negotiation.cc:407] Connection header received
0926 04:03:11.299543 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.299544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.299558 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.299619 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.300280 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.300808 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.301938 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.302166 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.302231 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.304988 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.304990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.304990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.304999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.307085 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.307607 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.307608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.307608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.307681 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.307955 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.307956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.307956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.308081 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.308155 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.308449 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.308549 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:03:11.308841 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.298076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37675 (local address 127.0.200.129:42571)
0926 04:03:11.298284 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:11.298330 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:11.299430 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:11.299685 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:11.299688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:11.299955 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:11.300165 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.300169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.300980 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.300981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.301838 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.301840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.302286 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.302287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.302408 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.303039 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:11.303050 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:11.304864 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:11.307213 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.307223 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.307226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.307499 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.307750 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.307751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.307752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.307872 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.308157 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:11.308164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:11.308313 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 04:03:11.308427 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 04:03:11.308600 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":128,"threads_started":1}
I20260926 04:03:11.309995 4578 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37675
I20260926 04:03:11.310160 4578 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:11.310355 4578 heartbeater.cc:511] Master 127.0.200.190:37675 requested a full tablet report, sending...
I20260926 04:03:11.310895 4387 ts_manager.cc:194] Registered new tserver with Master: 0f37a53d5caf44b9902a8306601937db (127.0.200.129:34003)
I20260926 04:03:11.311535 4387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:42571
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:11.339527 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:11.426350 4583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:11.426656 4583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:11.426702 4583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:11.427994 4583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:11.428090 4583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:11.428138 4583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:11.429829 4583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:11.430160 4583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:11.430413 4583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:11.436254 4591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.436333 4583 server_base.cc:1034] running on GCE node
W20260926 04:03:11.436395 4589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:11.436522 4588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.436790 4583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:11.437085 4583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:11.438280 4583 hybrid_clock.cc:648] HybridClock initialized: now 1790395391438246 us; error 55 us; skew 500 ppm
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:11.441826 4583 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:11.442353 4583 webserver.cc:460] Webserver started at http://127.0.200.130:45379/ using document root <none> and password file <none>
I20260926 04:03:11.442625 4583 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:11.442674 4583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:11.442770 4583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:11.443821 4583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "2f3f563f92b440f8a4e30942f4dec63e"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d2785f0911484adaf51239cacced5c8e"
server_key_iv: "4d51353dfb2f223e9a061dca4e5cb451"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.444139 4583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "2f3f563f92b440f8a4e30942f4dec63e"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d2785f0911484adaf51239cacced5c8e"
server_key_iv: "4d51353dfb2f223e9a061dca4e5cb451"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.445611 4583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:11.446537 4598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.446794 4583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:11.446904 4583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "2f3f563f92b440f8a4e30942f4dec63e"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "d2785f0911484adaf51239cacced5c8e"
server_key_iv: "4d51353dfb2f223e9a061dca4e5cb451"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.446996 4583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:11.464223 4583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:11.466833 4583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:11.467010 4583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:11.467262 4583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:11.467590 4583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:11.467651 4583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.467694 4583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:11.467723 4583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.474069 4583 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:45123
I20260926 04:03:11.474117 4711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:45123 every 8 connection(s)
I20260926 04:03:11.474436 4583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:11.475449 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4583
I20260926 04:03:11.475582 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:11.475792 802 external_mini_cluster.cc:1428] Setting key f85275233b6260f0df3813e0e6c776a4
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:03:11.485476 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.475297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:50939 (local address 127.0.200.190:37675)
0926 04:03:11.475999 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.476001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.476012 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:11.476805 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.476806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.476823 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.476930 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.477575 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.478126 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.479358 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.479558 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.479646 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.482247 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.482249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.482249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.482257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.483923 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.484411 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.484412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.484413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.484475 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.484736 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.484737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.484737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.484857 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.484950 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.485292 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.485380 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677}
I20260926 04:03:11.485698 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.475187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37675 (local address 127.0.200.130:50939)
0926 04:03:11.475374 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:11.475414 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:11.476624 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:11.476990 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:11.476993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:11.477261 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:11.477458 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.477463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.478233 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.478234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.479256 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.479259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.479654 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.479655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.479793 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.480394 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:11.480403 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:11.482126 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:11.484072 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.484081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.484084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.484328 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.484542 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.484543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.484543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.484657 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.484944 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:11.484950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:11.485139 (+ 189us) client_negotiation.cc:769] Sending connection context
0926 04:03:11.485261 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 04:03:11.485435 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260926 04:03:11.486395 4712 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37675
I20260926 04:03:11.486604 4712 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:11.486833 4712 heartbeater.cc:511] Master 127.0.200.190:37675 requested a full tablet report, sending...
I20260926 04:03:11.487396 4387 ts_manager.cc:194] Registered new tserver with Master: 2f3f563f92b440f8a4e30942f4dec63e (127.0.200.130:45123)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
I20260926 04:03:11.487870 4387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:50939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:11.506055 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:11.590359 4717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:11.590651 4717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:11.590693 4717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:11.591837 4717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:11.591874 4717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:11.591899 4717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:11.593317 4717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:11.593573 4717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:11.593818 4717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:11.598961 4725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:11.599260 4723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.599339 4717 server_base.cc:1034] running on GCE node
W20260926 04:03:11.599519 4722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:11.599785 4717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:11.600088 4717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:11.601305 4717 hybrid_clock.cc:648] HybridClock initialized: now 1790395391601239 us; error 87 us; skew 500 ppm
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:11.604110 4717 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:11.604605 4717 webserver.cc:460] Webserver started at http://127.0.200.131:34501/ using document root <none> and password file <none>
I20260926 04:03:11.604795 4717 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:11.604838 4717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:11.604933 4717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:11.606009 4717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "c6bfcf250efd4d1fb7a5a93a9e3ad098"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "483d18000e23a23ce80104df812892a8"
server_key_iv: "303b63dc1f8af2674819a06bb1a4f8fb"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.606328 4717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "c6bfcf250efd4d1fb7a5a93a9e3ad098"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "483d18000e23a23ce80104df812892a8"
server_key_iv: "303b63dc1f8af2674819a06bb1a4f8fb"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.607759 4717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:11.608603 4732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.608788 4717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:11.608902 4717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "c6bfcf250efd4d1fb7a5a93a9e3ad098"
format_stamp: "Formatted at 2026-09-26 04:03:11 on dist-test-slave-j8c1"
server_key: "483d18000e23a23ce80104df812892a8"
server_key_iv: "303b63dc1f8af2674819a06bb1a4f8fb"
server_key_version: "encryptionkey@0"
I20260926 04:03:11.608986 4717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:11.619783 4717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:11.622009 4717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:11.622143 4717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:11.622398 4717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:11.622777 4717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:11.622822 4717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.622846 4717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:11.622860 4717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:11.628827 4717 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:42137
I20260926 04:03:11.628958 4845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:42137 every 8 connection(s)
I20260926 04:03:11.629307 4717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:11.631340 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4717
I20260926 04:03:11.631438 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:11.631626 802 external_mini_cluster.cc:1428] Setting key 6217322a24098816c22b2ef5ab02b882
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:11.639721 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.630175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:38819 (local address 127.0.200.190:37675)
0926 04:03:11.630258 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.630260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.630461 (+ 201us) server_negotiation.cc:407] Connection header received
0926 04:03:11.631800 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.631802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.631818 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.631869 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.632580 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.633071 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.634146 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.634334 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.634398 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.636840 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.636842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.636842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.636849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.638311 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.638823 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.638824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.638825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.638877 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.639138 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.639139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.639139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.639243 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.639291 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.639591 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.639653 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 04:03:11.639874 4848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.630146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37675 (local address 127.0.200.131:38819)
0926 04:03:11.630324 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:11.630363 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:11.631619 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:11.631949 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:11.631954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:11.632304 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:11.632507 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.632512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.633155 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.633156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.634046 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:11.634048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.634430 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.634431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:11.634602 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.635239 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:11.635247 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:11.636740 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:11.638395 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.638401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.638403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.638727 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.638960 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:11.638961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:11.638961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:11.639072 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:11.639327 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:11.639332 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:11.639459 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 04:03:11.639555 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 04:03:11.639693 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260926 04:03:11.640488 4846 heartbeater.cc:347] Connected to a master server at 127.0.200.190:37675
I20260926 04:03:11.640602 4846 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:11.640764 4846 heartbeater.cc:511] Master 127.0.200.190:37675 requested a full tablet report, sending...
I20260926 04:03:11.641104 4387 ts_manager.cc:194] Registered new tserver with Master: c6bfcf250efd4d1fb7a5a93a9e3ad098 (127.0.200.131:42137)
I20260926 04:03:11.641546 4387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:38819
I20260926 04:03:11.643332 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:03:11.668996 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.659971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.0.200.129:34003)
0926 04:03:11.660126 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.660132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.660152 (+ 20us) server_negotiation.cc:407] Connection header received
0926 04:03:11.660178 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.660180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.660193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.660265 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.660899 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.661673 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.662422 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.662796 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.662887 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.665853 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.665863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.665863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.665871 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.667619 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.668096 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.668099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.668099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.668161 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.668428 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.668428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.668429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.668593 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.668703 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.668831 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.668901 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:11.679663 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:11.670649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.0.200.190:37675)
0926 04:03:11.670709 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:11.670711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:11.670720 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:11.670772 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:11.670772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:11.670781 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:11.670829 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:11.671316 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.671901 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.672617 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:11.672865 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:11.672934 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:11.675895 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:11.675897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:11.675897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:11.675906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:11.677828 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.678451 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.678453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.678454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.678534 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:11.678874 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:11.678875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:11.678875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:11.679090 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:11.679179 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:11.679403 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 04:03:11.679497 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 04:03:11.679924 4387 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40556
I20260926 04:03:11.680346 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4449
I20260926 04:03:11.685067 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4583
I20260926 04:03:11.689425 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4717
I20260926 04:03:11.694168 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4355
2026-09-26T04:03:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1961 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4858](info): set up 2 sockets
Sep 26 04:03:11 dist-test-slave-j8c1 krb5kdc[4858](info): commencing operation
krb5kdc: starting...
W20260926 04:03:12.750622 802 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:12 dist-test-slave-j8c1 krb5kdc[4858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:12.879787 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33511
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:12.962265 4871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:12.962535 4871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:12.962604 4871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:12.963919 4871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:12.964002 4871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:12.964028 4871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:12.964057 4871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:12.964087 4871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:12.965718 4871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:12.966030 4871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:12.966279 4871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:12.971487 4876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:12.971536 4877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:12.971621 4879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:12.971859 4871 server_base.cc:1034] running on GCE node
I20260926 04:03:12.972178 4871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:12.972525 4871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:12.973685 4871 hybrid_clock.cc:648] HybridClock initialized: now 1790395392973622 us; error 87 us; skew 500 ppm
Sep 26 04:03:12 dist-test-slave-j8c1 krb5kdc[4858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:12.976799 4871 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:12.977236 4871 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:12.977290 4871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:12.977432 4871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:12.978679 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "192a7af14c624358a081d179fb9635e3"
format_stamp: "Formatted at 2026-09-26 04:03:12 on dist-test-slave-j8c1"
server_key: "01344b488b1ea9d23af207532cb1cf5b"
server_key_iv: "2a728cd69b58a130b6882369a9f17307"
server_key_version: "encryptionkey@0"
I20260926 04:03:12.979084 4871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "192a7af14c624358a081d179fb9635e3"
format_stamp: "Formatted at 2026-09-26 04:03:12 on dist-test-slave-j8c1"
server_key: "01344b488b1ea9d23af207532cb1cf5b"
server_key_iv: "2a728cd69b58a130b6882369a9f17307"
server_key_version: "encryptionkey@0"
I20260926 04:03:12.980486 4871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:03:12.981419 4885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:12.981670 4871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:12.981758 4871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "192a7af14c624358a081d179fb9635e3"
format_stamp: "Formatted at 2026-09-26 04:03:12 on dist-test-slave-j8c1"
server_key: "01344b488b1ea9d23af207532cb1cf5b"
server_key_iv: "2a728cd69b58a130b6882369a9f17307"
server_key_version: "encryptionkey@0"
I20260926 04:03:12.981859 4871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:12.991779 4871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:12.994005 4871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:12.994124 4871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:12.998453 4871 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:33511
I20260926 04:03:12.998490 4937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:33511 every 8 connection(s)
I20260926 04:03:12.998835 4871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:12.999835 4938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:13.002095 4938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: Bootstrap starting.
I20260926 04:03:13.002738 4938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:13.002981 4938 log.cc:824] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:13.003603 4938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: No bootstrap required, opened a new log
I20260926 04:03:13.004917 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4871
I20260926 04:03:13.004999 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:13.005204 802 external_mini_cluster.cc:1428] Setting key 2b1e6162a13483f810d82d79069be571
I20260926 04:03:13.005359 4938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } }
I20260926 04:03:13.005565 4938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:13.005611 4938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 192a7af14c624358a081d179fb9635e3, State: Initialized, Role: FOLLOWER
I20260926 04:03:13.005983 4938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } }
I20260926 04:03:13.006088 4938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:13.006136 4938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:13.006196 4938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:13.006855 4938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } }
I20260926 04:03:13.006974 4938 leader_election.cc:302] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 192a7af14c624358a081d179fb9635e3; no voters:
I20260926 04:03:13.007148 4938 leader_election.cc:288] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:13.007184 4943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:13.007354 4943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [term 1 LEADER]: Becoming Leader. State: Replica: 192a7af14c624358a081d179fb9635e3, State: Running, Role: LEADER
I20260926 04:03:13.007429 4938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:13.007452 4943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } }
I20260926 04:03:13.007737 4943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 192a7af14c624358a081d179fb9635e3. Latest consensus state: current_term: 1 leader_uuid: "192a7af14c624358a081d179fb9635e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } } }
I20260926 04:03:13.007824 4943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:13.007747 4945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "192a7af14c624358a081d179fb9635e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192a7af14c624358a081d179fb9635e3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33511 } } }
I20260926 04:03:13.007987 4945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:13.008029 4948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:13.008553 4948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:13.010704 4948 catalog_manager.cc:1347] Generated new cluster ID: 6523030ec52244999135bd196fe013ae
I20260926 04:03:13.010785 4948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:13.018006 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.005789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.0.200.190:33511)
0926 04:03:13.006005 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.006018 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.006045 (+ 27us) server_negotiation.cc:407] Connection header received
0926 04:03:13.006540 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.006595 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.006662 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.006795 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.007633 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.009148 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.010113 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.010744 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.010859 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.013809 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.013814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.013815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:13.013829 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.016121 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.016532 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.016536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.016536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.016674 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.016885 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.016886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.016886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.017256 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.017433 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.017508 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.017773 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260926 04:03:13.024536 4948 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:13.025336 4948 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:13.030336 4948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: Generated new TSK 0
I20260926 04:03:13.030586 4948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:13.062525 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:13.147791 4964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:13.148054 4964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:13.148099 4964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:13.149394 4964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:13.149478 4964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:13.149529 4964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:13.151297 4964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:13.151610 4964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:13.151868 4964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:13.157241 4970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.157279 4969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.157241 4972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.157645 4964 server_base.cc:1034] running on GCE node
I20260926 04:03:13.157876 4964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:13.158157 4964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:13.159322 4964 hybrid_clock.cc:648] HybridClock initialized: now 1790395393159306 us; error 28 us; skew 500 ppm
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:13.162391 4964 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:13.162837 4964 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:13.162915 4964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:13.163072 4964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:13.164247 4964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "bc6e39e972d74d39b1f9fa48f289f1bc"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "184eda0d8b15d7b610394e6f9c57906a"
server_key_iv: "0da1e342510a87127048b5098992115c"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.164624 4964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "bc6e39e972d74d39b1f9fa48f289f1bc"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "184eda0d8b15d7b610394e6f9c57906a"
server_key_iv: "0da1e342510a87127048b5098992115c"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.166126 4964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:13.167053 4978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.167318 4964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:03:13.167431 4964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "bc6e39e972d74d39b1f9fa48f289f1bc"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "184eda0d8b15d7b610394e6f9c57906a"
server_key_iv: "0da1e342510a87127048b5098992115c"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.167515 4964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:13.179800 4964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:13.182426 4964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:13.182675 4964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:13.182950 4964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:13.183321 4964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:13.183383 4964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.183425 4964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:13.183455 4964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.190326 4964 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:38251
I20260926 04:03:13.190358 5091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:38251 every 8 connection(s)
I20260926 04:03:13.190754 4964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:13.198174 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 4964
I20260926 04:03:13.198290 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:13.198487 802 external_mini_cluster.cc:1428] Setting key 3264f027a13ffd9c3a136445b67dba40
I20260926 04:03:13.202828 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.191612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:43149 (local address 127.0.200.190:33511)
0926 04:03:13.191738 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.191741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.191913 (+ 172us) server_negotiation.cc:407] Connection header received
0926 04:03:13.192505 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.192506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.192518 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.192576 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.193751 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.194276 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.195402 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.195612 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.195708 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.198313 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.198315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.198315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:13.198324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.200977 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.201537 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.201538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.201539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.201608 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.201945 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.201946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.201946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.202112 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.202199 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.202630 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.202730 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 04:03:13.202977 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.191565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33511 (local address 127.0.200.129:43149)
0926 04:03:13.191765 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.191816 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.192389 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.192671 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.192675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.192945 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:13.193637 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.193646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.194380 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.194381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.195304 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.195307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.195722 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.195722 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.195856 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.196447 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:13.196456 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:13.198148 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:13.201082 (+ 2934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.201092 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.201096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.201445 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.201696 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.201698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.201698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.201864 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.202186 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:13.202194 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:13.202373 (+ 179us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.202503 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.202727 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260926 04:03:13.204941 5092 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33511
I20260926 04:03:13.205070 5092 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:13.205286 5092 heartbeater.cc:511] Master 127.0.200.190:33511 requested a full tablet report, sending...
I20260926 04:03:13.205922 4902 ts_manager.cc:194] Registered new tserver with Master: bc6e39e972d74d39b1f9fa48f289f1bc (127.0.200.129:38251)
I20260926 04:03:13.206983 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:43149
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:13.230861 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:13.314258 5097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:13.314498 5097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:13.314527 5097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:13.315939 5097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:13.315995 5097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:13.316027 5097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:13.317715 5097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:13.318005 5097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:13.318265 5097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:13.323580 5103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.323668 5097 server_base.cc:1034] running on GCE node
W20260926 04:03:13.323614 5105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.323580 5102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.324015 5097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:13.324298 5097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:13.325477 5097 hybrid_clock.cc:648] HybridClock initialized: now 1790395393325454 us; error 42 us; skew 500 ppm
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:13.328265 5097 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:13.328715 5097 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:13.328785 5097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:13.328924 5097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:13.330021 5097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "d195cc0145e1464199b1ab719031a33b"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "e400e37c53dbd4281790af1adc86eb32"
server_key_iv: "11b16f84ecadf95dbe8fcc828776ded3"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.330397 5097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "d195cc0145e1464199b1ab719031a33b"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "e400e37c53dbd4281790af1adc86eb32"
server_key_iv: "11b16f84ecadf95dbe8fcc828776ded3"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.331842 5097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:13.332652 5111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.332878 5097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:03:13.332976 5097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "d195cc0145e1464199b1ab719031a33b"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "e400e37c53dbd4281790af1adc86eb32"
server_key_iv: "11b16f84ecadf95dbe8fcc828776ded3"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.333058 5097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:13.347563 5097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:13.349879 5097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:13.350063 5097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:13.350363 5097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:13.350732 5097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:13.350792 5097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.350834 5097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:13.350876 5097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.357259 5097 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:40287
I20260926 04:03:13.357295 5224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:40287 every 8 connection(s)
I20260926 04:03:13.357607 5097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:13.366263 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5097
I20260926 04:03:13.366350 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:13.366586 802 external_mini_cluster.cc:1428] Setting key ce2ac95679f1fe023dba8530f6acc118
I20260926 04:03:13.367658 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.358426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:42139 (local address 127.0.200.190:33511)
0926 04:03:13.358495 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.358497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.358745 (+ 248us) server_negotiation.cc:407] Connection header received
0926 04:03:13.359263 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.359264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.359275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.359337 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.360255 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.360778 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.361844 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.362035 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.362090 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.364558 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.364560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.364560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:13.364568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.365941 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.366486 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.366488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.366488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.366594 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.366881 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.366882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.366883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.367042 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.367121 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.367515 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.367590 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:03:13.368057 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.358390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33511 (local address 127.0.200.130:42139)
0926 04:03:13.358608 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.358655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.359180 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.359401 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.359405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.359672 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:13.360169 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.360175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.360876 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.360877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.361758 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.361760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.362152 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.362154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.362255 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.362884 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:13.362894 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:13.364415 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:13.366051 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.366058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.366061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.366363 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.366667 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.366668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.366668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.366811 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.367140 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:13.367145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:13.367348 (+ 203us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.367500 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.367716 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":115,"threads_started":1}
I20260926 04:03:13.368582 5225 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33511
I20260926 04:03:13.368693 5225 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:13.368880 5225 heartbeater.cc:511] Master 127.0.200.190:33511 requested a full tablet report, sending...
I20260926 04:03:13.369352 4902 ts_manager.cc:194] Registered new tserver with Master: d195cc0145e1464199b1ab719031a33b (127.0.200.130:40287)
I20260926 04:03:13.369903 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:42139
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:13.394955 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:13.478113 5230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:13.478397 5230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:13.478440 5230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:13.480370 5230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:13.480475 5230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:13.480633 5230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:13.482785 5230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:13.483080 5230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:13.483315 5230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:13.488765 5236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.488770 5238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.488768 5235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.489003 5230 server_base.cc:1034] running on GCE node
I20260926 04:03:13.489269 5230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:13.489578 5230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:13.490768 5230 hybrid_clock.cc:648] HybridClock initialized: now 1790395393490761 us; error 34 us; skew 500 ppm
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:13.493484 5230 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:13.493937 5230 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:13.493988 5230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:13.494129 5230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:13.495378 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "4091c90dd8024b40b5305dfd5aebe504"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "baf9116a67c847ddbe067172dc853c00"
server_key_iv: "747c96824f7a4478347024490f997cb4"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.495750 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "4091c90dd8024b40b5305dfd5aebe504"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "baf9116a67c847ddbe067172dc853c00"
server_key_iv: "747c96824f7a4478347024490f997cb4"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.497085 5230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:13.497876 5244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.498066 5230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:13.498153 5230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "4091c90dd8024b40b5305dfd5aebe504"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "baf9116a67c847ddbe067172dc853c00"
server_key_iv: "747c96824f7a4478347024490f997cb4"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.498262 5230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:13.515874 5230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:13.518076 5230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:13.518229 5230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:13.518425 5230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:13.518781 5230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:13.518816 5230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.518872 5230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:13.518899 5230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.524801 5230 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:44893
I20260926 04:03:13.524832 5357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:44893 every 8 connection(s)
I20260926 04:03:13.525127 5230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:13.530427 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5230
I20260926 04:03:13.530539 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:13.530759 802 external_mini_cluster.cc:1428] Setting key 90d33b404de26df7942c5b58f6af162a
Sep 26 04:03:13 dist-test-slave-j8c1 krb5kdc[4858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:13.535490 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.525972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:45993 (local address 127.0.200.190:33511)
0926 04:03:13.526048 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.526050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.526206 (+ 156us) server_negotiation.cc:407] Connection header received
0926 04:03:13.526872 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.526873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.526883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.526944 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.527939 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.528467 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.529500 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.529686 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.529751 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.532538 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.532539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.532540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:13.532546 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.534083 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.534469 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.534470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.534471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.534522 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.534801 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.534802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.534802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.534909 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.534963 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.535344 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.535423 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 04:03:13.535677 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.525862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33511 (local address 127.0.200.131:45993)
0926 04:03:13.526056 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.526096 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.526748 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.527017 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.527021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.527332 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:13.527819 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.527824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.528578 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.528580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.529401 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.529403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.529771 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.529772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.529873 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.530660 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:13.530670 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:13.532432 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:13.534173 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.534179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.534181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.534402 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.534630 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:13.534631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:13.534632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:13.534745 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:13.534990 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:13.534995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:13.535129 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.535238 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.535385 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260926 04:03:13.536173 5358 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33511
I20260926 04:03:13.536283 5358 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:13.536432 5358 heartbeater.cc:511] Master 127.0.200.190:33511 requested a full tablet report, sending...
I20260926 04:03:13.536908 4902 ts_manager.cc:194] Registered new tserver with Master: 4091c90dd8024b40b5305dfd5aebe504 (127.0.200.131:44893)
I20260926 04:03:13.537343 4902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:45993
I20260926 04:03:13.542888 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:03:13.551101 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.544444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.0.200.190:33511)
0926 04:03:13.544485 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.544487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.544630 (+ 143us) server_negotiation.cc:407] Connection header received
0926 04:03:13.544685 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.544686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.544694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.544746 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.545464 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.545954 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.546769 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.546952 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.547018 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.548277 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.548278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.548278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:13.548285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.549757 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.550214 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.550216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.550216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.550275 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:13.550483 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:13.550484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:13.550484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:13.550708 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.550809 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.550962 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.551028 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:13.552906 4902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:13.554500 4902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:13.561497 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41530 (local address 127.0.200.130:40287)
0926 04:03:13.557490 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.557498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.557608 (+ 110us) server_negotiation.cc:407] Connection header received
0926 04:03:13.557648 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.557650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.557664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.557731 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:13.557997 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559162 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.560555 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561020 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561107 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561163 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.561360 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.561407 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":81,"threads_started":1}
I20260926 04:03:13.561569 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44893 (local address 127.0.0.1:34134)
0926 04:03:13.557617 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.557628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.557687 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.558000 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.558001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.558003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:13.558050 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.558052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559127 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.559128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.560452 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.560455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561320 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.561375 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561378 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.561426 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.561463 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260926 04:03:13.561635 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.0.200.129:38251)
0926 04:03:13.557570 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.557575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.557585 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:13.557629 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.557630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.557648 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.557727 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:13.557998 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559191 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.560335 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561023 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561096 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561153 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.561516 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.561555 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260926 04:03:13.561770 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:38251 (local address 127.0.0.1:35300)
0926 04:03:13.557196 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.557245 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.557309 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.557824 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.557827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.557829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:13.557910 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.557913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559293 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.559294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.560243 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.560245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561189 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.561254 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561259 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.561309 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.561401 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":6,"thread_start_us":65,"threads_started":1}
I20260926 04:03:13.561875 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40287 (local address 127.0.0.1:41530)
0926 04:03:13.557466 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.557478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.557549 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.557824 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.557827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.557833 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:13.557910 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.557914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559274 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.559275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.560453 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.560455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561121 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.561207 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561222 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.561310 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.561413 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":97,"thread_start_us":90,"threads_started":1}
I20260926 04:03:13.561960 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.557591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.0.200.131:44893)
0926 04:03:13.557785 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.557789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.557801 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:13.557841 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.557843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.557897 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.558122 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:13.558203 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.559024 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.560543 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.561232 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.561305 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.561357 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.561515 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.561553 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260926 04:03:13.562620 5026 tablet_service.cc:1467] Processing CreateTablet for tablet e3c496467a614182862626fae6a7363f (DEFAULT_TABLE table=test-table [id=33f6188df894410489119a0686daa8cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:13.562625 5292 tablet_service.cc:1467] Processing CreateTablet for tablet e3c496467a614182862626fae6a7363f (DEFAULT_TABLE table=test-table [id=33f6188df894410489119a0686daa8cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:13.562968 5292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c496467a614182862626fae6a7363f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:13.562973 5026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c496467a614182862626fae6a7363f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:13.565662 5376 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Bootstrap starting.
I20260926 04:03:13.565716 5375 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: Bootstrap starting.
I20260926 04:03:13.565660 5159 tablet_service.cc:1467] Processing CreateTablet for tablet e3c496467a614182862626fae6a7363f (DEFAULT_TABLE table=test-table [id=33f6188df894410489119a0686daa8cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:13.566018 5159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c496467a614182862626fae6a7363f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:13.566381 5376 tablet_bootstrap.cc:654] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:13.566412 5375 tablet_bootstrap.cc:654] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:13.566956 5376 log.cc:824] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:13.567066 5375 log.cc:824] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:13.567783 5375 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: No bootstrap required, opened a new log
I20260926 04:03:13.567829 5376 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: No bootstrap required, opened a new log
I20260926 04:03:13.567873 5375 ts_tablet_manager.cc:1397] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:13.567919 5376 ts_tablet_manager.cc:1397] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:13.569342 5375 raft_consensus.cc:348] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.569510 5375 raft_consensus.cc:374] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:13.569557 5375 raft_consensus.cc:729] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e39e972d74d39b1f9fa48f289f1bc, State: Initialized, Role: FOLLOWER
I20260926 04:03:13.569666 5375 consensus_queue.cc:260] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.570022 5375 ts_tablet_manager.cc:1428] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:13.570148 5092 heartbeater.cc:503] Master 127.0.200.190:33511 was elected leader, sending a full tablet report...
I20260926 04:03:13.570339 5376 raft_consensus.cc:348] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.570501 5379 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Bootstrap starting.
I20260926 04:03:13.570537 5376 raft_consensus.cc:374] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:13.570626 5376 raft_consensus.cc:729] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091c90dd8024b40b5305dfd5aebe504, State: Initialized, Role: FOLLOWER
I20260926 04:03:13.570739 5376 consensus_queue.cc:260] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.571059 5358 heartbeater.cc:503] Master 127.0.200.190:33511 was elected leader, sending a full tablet report...
I20260926 04:03:13.571313 5376 ts_tablet_manager.cc:1428] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260926 04:03:13.571352 5379 tablet_bootstrap.cc:654] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:13.572053 5379 log.cc:824] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:13.573066 5379 tablet_bootstrap.cc:492] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: No bootstrap required, opened a new log
I20260926 04:03:13.573156 5379 ts_tablet_manager.cc:1397] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:03:13.574595 5379 raft_consensus.cc:348] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.574769 5379 raft_consensus.cc:374] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:13.574813 5379 raft_consensus.cc:729] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d195cc0145e1464199b1ab719031a33b, State: Initialized, Role: FOLLOWER
I20260926 04:03:13.574936 5379 consensus_queue.cc:260] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.575232 5379 ts_tablet_manager.cc:1428] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:13.575233 5225 heartbeater.cc:503] Master 127.0.200.190:33511 was elected leader, sending a full tablet report...
I20260926 04:03:13.600121 5381 raft_consensus.cc:482] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:13.600247 5381 raft_consensus.cc:504] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.600600 5381 leader_election.cc:288] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6e39e972d74d39b1f9fa48f289f1bc (127.0.200.129:38251), d195cc0145e1464199b1ab719031a33b (127.0.200.130:40287)
I20260926 04:03:13.603615 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.600907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:36477 (local address 127.0.200.130:40287)
0926 04:03:13.600953 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.600955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.600963 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:13.600988 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.600989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.601006 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.601078 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:13.601254 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.601726 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.602871 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.603244 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.603335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.603345 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.603501 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.603541 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:13.603615 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.600795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40287 (local address 127.0.200.131:36477)
0926 04:03:13.600858 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.600872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.600913 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.601082 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.601083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.601084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:13.601165 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.601167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.601815 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.601816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.602780 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.602782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.603335 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.603336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.603421 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.603424 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.603494 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.603537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 04:03:13.603826 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.600867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:49169 (local address 127.0.200.129:38251)
0926 04:03:13.600931 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.600933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.601077 (+ 144us) server_negotiation.cc:407] Connection header received
0926 04:03:13.601198 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.601200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.601210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.601261 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:13.601478 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.601996 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.603050 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.603415 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.603499 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.603507 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.603674 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.603725 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 04:03:13.604012 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.600818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:38251 (local address 127.0.200.131:49169)
0926 04:03:13.600995 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.601012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.601113 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.601263 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.601264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.601266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:13.601406 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.601407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.602112 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.602113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.602949 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:13.602951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.603510 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.603511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:13.603590 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.603593 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.603666 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.603860 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260926 04:03:13.603955 5179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c496467a614182862626fae6a7363f" candidate_uuid: "4091c90dd8024b40b5305dfd5aebe504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d195cc0145e1464199b1ab719031a33b" is_pre_election: true
I20260926 04:03:13.604162 5179 raft_consensus.cc:2454] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091c90dd8024b40b5305dfd5aebe504 in term 0.
I20260926 04:03:13.604362 5046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c496467a614182862626fae6a7363f" candidate_uuid: "4091c90dd8024b40b5305dfd5aebe504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" is_pre_election: true
I20260926 04:03:13.604417 5246 leader_election.cc:302] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091c90dd8024b40b5305dfd5aebe504, d195cc0145e1464199b1ab719031a33b; no voters:
I20260926 04:03:13.604497 5046 raft_consensus.cc:2454] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091c90dd8024b40b5305dfd5aebe504 in term 0.
I20260926 04:03:13.604545 5381 raft_consensus.cc:2781] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:13.604607 5381 raft_consensus.cc:482] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:13.604640 5381 raft_consensus.cc:3037] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:13.605388 5381 raft_consensus.cc:504] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
I20260926 04:03:13.605574 5381 leader_election.cc:288] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [CANDIDATE]: Term 1 election: Requested vote from peers bc6e39e972d74d39b1f9fa48f289f1bc (127.0.200.129:38251), d195cc0145e1464199b1ab719031a33b (127.0.200.130:40287)
I20260926 04:03:13.605733 5179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c496467a614182862626fae6a7363f" candidate_uuid: "4091c90dd8024b40b5305dfd5aebe504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d195cc0145e1464199b1ab719031a33b"
I20260926 04:03:13.605788 5046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c496467a614182862626fae6a7363f" candidate_uuid: "4091c90dd8024b40b5305dfd5aebe504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc"
I20260926 04:03:13.605835 5179 raft_consensus.cc:3037] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:13.605913 5046 raft_consensus.cc:3037] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:13.606666 5179 raft_consensus.cc:2454] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091c90dd8024b40b5305dfd5aebe504 in term 1.
I20260926 04:03:13.606823 5246 leader_election.cc:302] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091c90dd8024b40b5305dfd5aebe504, d195cc0145e1464199b1ab719031a33b; no voters:
I20260926 04:03:13.606951 5381 raft_consensus.cc:2781] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:13.607014 5046 raft_consensus.cc:2454] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091c90dd8024b40b5305dfd5aebe504 in term 1.
I20260926 04:03:13.607133 5381 raft_consensus.cc:686] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 1 LEADER]: Becoming Leader. State: Replica: 4091c90dd8024b40b5305dfd5aebe504, State: Running, Role: LEADER
I20260926 04:03:13.607234 5381 consensus_queue.cc:237] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } }
W20260926 04:03:13.608093 5226 tablet.cc:2369] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:13.608106 4902 catalog_manager.cc:5495] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4091c90dd8024b40b5305dfd5aebe504 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "4091c90dd8024b40b5305dfd5aebe504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091c90dd8024b40b5305dfd5aebe504" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44893 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:13.624676 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.621922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.0.200.131:44893)
0926 04:03:13.621989 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.621991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.622002 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:13.622095 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.622096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.622117 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.622243 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:13.622347 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.622903 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.623819 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:13.624021 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:13.624103 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:13.624298 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:13.624408 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:13.624511 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.624529 (+ 18us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.624551 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:03:13.626951 5046 raft_consensus.cc:1261] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 1 FOLLOWER]: Refusing update from remote peer 4091c90dd8024b40b5305dfd5aebe504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:13.626991 5179 raft_consensus.cc:1261] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 1 FOLLOWER]: Refusing update from remote peer 4091c90dd8024b40b5305dfd5aebe504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:13.627427 5381 consensus_queue.cc:1035] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6e39e972d74d39b1f9fa48f289f1bc" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 38251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:13.627593 5385 consensus_queue.cc:1035] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d195cc0145e1464199b1ab719031a33b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:13.630884 5389 mvcc.cc:205] Tried to move back new op lower bound from 7333459532292435968 to 7333459532215971840. Current Snapshot: MvccSnapshot[applied={T|T < 7333459532292435968 or (T in {7333459532292435968})}]
I20260926 04:03:13.631072 5390 mvcc.cc:205] Tried to move back new op lower bound from 7333459532292435968 to 7333459532215971840. Current Snapshot: MvccSnapshot[applied={T|T < 7333459532292435968 or (T in {7333459532292435968})}]
I20260926 04:03:13.631276 5388 mvcc.cc:205] Tried to move back new op lower bound from 7333459532292435968 to 7333459532215971840. Current Snapshot: MvccSnapshot[applied={T|T < 7333459532292435968 or (T in {7333459532292435968})}]
I20260926 04:03:13.632618 4902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:13.632753 4902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:13.633687 4902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 192a7af14c624358a081d179fb9635e3: Sending DeleteTablet for 3 replicas of tablet e3c496467a614182862626fae6a7363f
I20260926 04:03:13.634074 5026 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c496467a614182862626fae6a7363f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:13 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35300
I20260926 04:03:13.634074 5159 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c496467a614182862626fae6a7363f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:13 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:41530
I20260926 04:03:13.634339 5397 tablet_replica.cc:330] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc: stopping tablet replica
I20260926 04:03:13.634452 5397 raft_consensus.cc:2229] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:13.634480 5292 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c496467a614182862626fae6a7363f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:13 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:34134
I20260926 04:03:13.634584 5397 raft_consensus.cc:2258] T e3c496467a614182862626fae6a7363f P bc6e39e972d74d39b1f9fa48f289f1bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:13.634719 5399 tablet_replica.cc:330] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: stopping tablet replica
I20260926 04:03:13.634714 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4964
I20260926 04:03:13.634805 5399 raft_consensus.cc:2229] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:13.635448 5399 raft_consensus.cc:2258] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:13.636401 5399 ts_tablet_manager.cc:1905] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:13.638502 5399 ts_tablet_manager.cc:1918] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:13.638610 5399 log.cc:1196] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/e3c496467a614182862626fae6a7363f
I20260926 04:03:13.638899 5399 ts_tablet_manager.cc:1939] T e3c496467a614182862626fae6a7363f P 4091c90dd8024b40b5305dfd5aebe504: Deleting consensus metadata
I20260926 04:03:13.639496 4886 catalog_manager.cc:4841] TS 4091c90dd8024b40b5305dfd5aebe504 (127.0.200.131:44893): tablet e3c496467a614182862626fae6a7363f (table test-table [id=33f6188df894410489119a0686daa8cf]) successfully deleted
W20260926 04:03:13.641232 4886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:38251 (error 108)
I20260926 04:03:13.641398 5398 tablet_replica.cc:330] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b: stopping tablet replica
I20260926 04:03:13.641510 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5097
I20260926 04:03:13.641528 5398 raft_consensus.cc:2229] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:13.641645 5398 raft_consensus.cc:2258] T e3c496467a614182862626fae6a7363f P d195cc0145e1464199b1ab719031a33b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:13.643198 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.643051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:38251 (local address 127.0.0.1:35308)
0926 04:03:13.643108 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.643151 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 04:03:13.643321 4886 catalog_manager.cc:4584] TS bc6e39e972d74d39b1f9fa48f289f1bc (127.0.200.129:38251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3c496467a614182862626fae6a7363f: Network error: Client connection negotiation failed: client connection to 127.0.200.129:38251: connect: Connection refused (error 111)
I20260926 04:03:13.648288 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.648143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40287 (local address 127.0.0.1:41536)
0926 04:03:13.648207 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.648238 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.130:40287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 04:03:13.648449 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5230
I20260926 04:03:13.653029 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 4871
2026-09-26T04:03:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1959 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T04:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:13Z Disabled control of system clock
I20260926 04:03:13.682076 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:38585
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:38585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 04:03:13.768894 5404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:13.769117 5404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:13.769141 5404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:13.770298 5404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:13.770339 5404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:13.770352 5404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:13.770365 5404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:13.770378 5404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:13.771811 5404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:38585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:38585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:13.772037 5404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:13.772306 5404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:13.777552 5410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.777540 5409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.777518 5412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.777786 5404 server_base.cc:1034] running on GCE node
I20260926 04:03:13.778110 5404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:13.778421 5404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:13.779580 5404 hybrid_clock.cc:648] HybridClock initialized: now 1790395393779569 us; error 31 us; skew 500 ppm
I20260926 04:03:13.780946 5404 webserver.cc:460] Webserver started at http://127.0.200.190:35885/ using document root <none> and password file <none>
I20260926 04:03:13.781176 5404 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:13.781221 5404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:13.781315 5404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:13.782421 5404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "9bfedd3ef91441d39129bd1d39c99271"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "75d299eff6ad624ae67af54bc6cd0011"
server_key_iv: "b94e93f73f0949efccf1b551f5438c04"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.782821 5404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "9bfedd3ef91441d39129bd1d39c99271"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "75d299eff6ad624ae67af54bc6cd0011"
server_key_iv: "b94e93f73f0949efccf1b551f5438c04"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.784137 5404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:13.784977 5418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.785224 5404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:13.785357 5404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "9bfedd3ef91441d39129bd1d39c99271"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "75d299eff6ad624ae67af54bc6cd0011"
server_key_iv: "b94e93f73f0949efccf1b551f5438c04"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.785452 5404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:13.789152 5404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:13.789456 5404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:13.789608 5404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:13.793836 5404 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:38585
I20260926 04:03:13.793895 5470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:38585 every 8 connection(s)
I20260926 04:03:13.794284 5404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:13.795089 5471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:13.796895 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5404
I20260926 04:03:13.797024 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:13.797237 802 external_mini_cluster.cc:1428] Setting key 5ff8b3c5dc874860cc50df61ece72a3b
I20260926 04:03:13.798277 5471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: Bootstrap starting.
I20260926 04:03:13.799014 5471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:13.799350 5471 log.cc:824] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:13.800006 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.797875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.0.200.190:38585)
0926 04:03:13.798065 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.798072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.798097 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:03:13.798542 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.798596 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.798655 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.798783 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.799543 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.799548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.799549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:13.799554 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.799613 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:13.799628 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.799737 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.799779 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.799839 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260926 04:03:13.800213 5471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: No bootstrap required, opened a new log
I20260926 04:03:13.801754 5471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } }
I20260926 04:03:13.801893 5471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:13.801916 5471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bfedd3ef91441d39129bd1d39c99271, State: Initialized, Role: FOLLOWER
I20260926 04:03:13.802037 5471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } }
I20260926 04:03:13.802120 5471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:13.802143 5471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:13.802171 5471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:13.803056 5471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } }
I20260926 04:03:13.803143 5471 leader_election.cc:302] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bfedd3ef91441d39129bd1d39c99271; no voters:
I20260926 04:03:13.803267 5471 leader_election.cc:288] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:13.803329 5476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:13.803508 5476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [term 1 LEADER]: Becoming Leader. State: Replica: 9bfedd3ef91441d39129bd1d39c99271, State: Running, Role: LEADER
I20260926 04:03:13.803576 5471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:13.803612 5476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } }
I20260926 04:03:13.804032 5477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bfedd3ef91441d39129bd1d39c99271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } } }
I20260926 04:03:13.804232 5478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bfedd3ef91441d39129bd1d39c99271. Latest consensus state: current_term: 1 leader_uuid: "9bfedd3ef91441d39129bd1d39c99271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bfedd3ef91441d39129bd1d39c99271" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38585 } } }
I20260926 04:03:13.804307 5477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:13.804332 5478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:13.804678 5486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:13.805236 5486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:13.806726 5486 catalog_manager.cc:1347] Generated new cluster ID: 34b0d96e55cd471c9f984babaa4f482c
I20260926 04:03:13.806777 5486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:13.832043 5486 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:13.832240 5486 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:13.852662 5486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: Generated new TSK 0
I20260926 04:03:13.852931 5486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:13.863556 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--builtin_ntp_servers=127.0.200.148:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 04:03:13.965871 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:13.966107 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:13.966133 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:13.967546 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:13.967599 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:13.967629 5495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:13.969147 5495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:13.969446 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:13.969769 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:13.974877 5503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.974931 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:13.974891 5501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:13.975147 5495 server_base.cc:1034] running on GCE node
I20260926 04:03:13.975397 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:13.975672 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:13.976881 5495 hybrid_clock.cc:648] HybridClock initialized: now 1790395393976873 us; error 36 us; skew 500 ppm
I20260926 04:03:13.978475 5495 webserver.cc:460] Webserver started at http://127.0.200.129:38365/ using document root <none> and password file <none>
I20260926 04:03:13.978793 5495 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:13.978873 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:13.979015 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:13.980154 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "b59eda72dfad433ba35d435951ac829d"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "1ee83963f8370357830349b7a41075b5"
server_key_iv: "c57cc90b6d36e322bc83ae1d19ec4c49"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.980582 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "b59eda72dfad433ba35d435951ac829d"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "1ee83963f8370357830349b7a41075b5"
server_key_iv: "c57cc90b6d36e322bc83ae1d19ec4c49"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.981994 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:13.983129 5509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.983477 5495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:13.983620 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b59eda72dfad433ba35d435951ac829d"
format_stamp: "Formatted at 2026-09-26 04:03:13 on dist-test-slave-j8c1"
server_key: "1ee83963f8370357830349b7a41075b5"
server_key_iv: "c57cc90b6d36e322bc83ae1d19ec4c49"
server_key_version: "encryptionkey@0"
I20260926 04:03:13.983707 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:13.987766 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:13.988095 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:13.988238 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:13.988448 5495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:13.988797 5495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:13.988853 5495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.988894 5495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:13.988924 5495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:13.994702 5495 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:43597
I20260926 04:03:13.994773 5622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:43597 every 8 connection(s)
I20260926 04:03:13.995119 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:13.998420 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.995988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:57423 (local address 127.0.200.190:38585)
0926 04:03:13.996072 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:13.996074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:13.996449 (+ 375us) server_negotiation.cc:407] Connection header received
0926 04:03:13.996805 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:13.996806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:13.996812 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:13.996905 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:13.998113 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:13.998114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:13.998114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:13.998118 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:13.998135 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:13.998141 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:13.998207 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:13.998350 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 04:03:13.998364 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 04:03:13.998502 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:13.995965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:38585 (local address 127.0.200.129:57423)
0926 04:03:13.996180 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:13.996213 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:13.996688 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:13.996960 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:13.996963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:13.996970 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:13.997936 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:13.997936 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:13.997979 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:13.998010 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:13.998220 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:13.998223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:13.998226 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:13.998288 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 04:03:13.998341 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260926 04:03:13.999061 5623 heartbeater.cc:347] Connected to a master server at 127.0.200.190:38585
I20260926 04:03:13.999168 5623 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:13.999392 5623 heartbeater.cc:511] Master 127.0.200.190:38585 requested a full tablet report, sending...
I20260926 04:03:13.999567 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5495
I20260926 04:03:13.999687 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:13.999859 802 external_mini_cluster.cc:1428] Setting key 34c21349d21d297da929639d8e3a5f9f
I20260926 04:03:14.000013 5435 ts_manager.cc:194] Registered new tserver with Master: b59eda72dfad433ba35d435951ac829d (127.0.200.129:43597)
I20260926 04:03:14.000905 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--builtin_ntp_servers=127.0.200.148:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 04:03:14.083943 5626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:14.084165 5626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:14.084190 5626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:14.085541 5626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:14.085594 5626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:14.085624 5626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:14.087541 5626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:14.087863 5626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:14.088274 5626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:14.093966 5631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.093999 5634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.093979 5632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.094410 5626 server_base.cc:1034] running on GCE node
I20260926 04:03:14.094725 5626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:14.095002 5626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:14.096148 5626 hybrid_clock.cc:648] HybridClock initialized: now 1790395394096141 us; error 34 us; skew 500 ppm
I20260926 04:03:14.104131 5626 webserver.cc:460] Webserver started at http://127.0.200.130:46033/ using document root <none> and password file <none>
I20260926 04:03:14.104481 5626 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:14.104537 5626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:14.104715 5626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:14.105898 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "879ac73abe894813ac59dcfce161f814"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "c60fe9c08b4d1ebf2c324efda94d5b51"
server_key_iv: "a9b1087f866822d22bf3e54aa3ae495a"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.106287 5626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "879ac73abe894813ac59dcfce161f814"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "c60fe9c08b4d1ebf2c324efda94d5b51"
server_key_iv: "a9b1087f866822d22bf3e54aa3ae495a"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.107995 5626 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 04:03:14.108896 5640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.109185 5626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:14.109272 5626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "879ac73abe894813ac59dcfce161f814"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "c60fe9c08b4d1ebf2c324efda94d5b51"
server_key_iv: "a9b1087f866822d22bf3e54aa3ae495a"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.109387 5626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:14.113147 5626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:14.113444 5626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:14.113593 5626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:14.113816 5626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:14.114176 5626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:14.114233 5626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.114274 5626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:14.114302 5626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.121208 5626 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:42435
I20260926 04:03:14.121250 5753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:42435 every 8 connection(s)
I20260926 04:03:14.121623 5626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:14.124927 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.122397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:41269 (local address 127.0.200.190:38585)
0926 04:03:14.122490 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.122494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.122733 (+ 239us) server_negotiation.cc:407] Connection header received
0926 04:03:14.123251 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.123253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.123259 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.123341 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.124642 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.124644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.124645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.124650 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.124668 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.124673 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.124737 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.124848 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.124868 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 04:03:14.125095 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.122381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:38585 (local address 127.0.200.130:41269)
0926 04:03:14.122607 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.122653 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.123133 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.123397 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.123401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.123411 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.124395 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.124395 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.124484 (+ 89us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.124528 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.124740 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.124743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.124746 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.124809 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.124871 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260926 04:03:14.125754 5754 heartbeater.cc:347] Connected to a master server at 127.0.200.190:38585
I20260926 04:03:14.125839 5754 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:14.126042 5754 heartbeater.cc:511] Master 127.0.200.190:38585 requested a full tablet report, sending...
I20260926 04:03:14.126174 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5626
I20260926 04:03:14.126315 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:14.126514 802 external_mini_cluster.cc:1428] Setting key ec25c3eaa1673495061864d78367717b
I20260926 04:03:14.126503 5434 ts_manager.cc:194] Registered new tserver with Master: 879ac73abe894813ac59dcfce161f814 (127.0.200.130:42435)
I20260926 04:03:14.127475 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--builtin_ntp_servers=127.0.200.148:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 04:03:14.210281 5757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:14.210521 5757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:14.210597 5757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:14.211829 5757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:14.211879 5757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:14.211908 5757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:14.213447 5757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:38585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:14.213721 5757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:14.214037 5757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:14.220261 5765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.220261 5763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.220491 5762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.220695 5757 server_base.cc:1034] running on GCE node
I20260926 04:03:14.220932 5757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:14.221339 5757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:14.222533 5757 hybrid_clock.cc:648] HybridClock initialized: now 1790395394222517 us; error 27 us; skew 500 ppm
I20260926 04:03:14.224807 5757 webserver.cc:460] Webserver started at http://127.0.200.131:39887/ using document root <none> and password file <none>
I20260926 04:03:14.225113 5757 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:14.225193 5757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:14.225346 5757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:14.227020 5757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "3359d3679fad479e8bc9933e48de870b"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "40f1698607c2160b4d272bfe635f089d"
server_key_iv: "a0fff6d4e71cc564821cd7f017543c87"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.227710 5757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "3359d3679fad479e8bc9933e48de870b"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "40f1698607c2160b4d272bfe635f089d"
server_key_iv: "a0fff6d4e71cc564821cd7f017543c87"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.229576 5757 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 04:03:14.230660 5771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.230875 5757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:14.231163 5757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "3359d3679fad479e8bc9933e48de870b"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "40f1698607c2160b4d272bfe635f089d"
server_key_iv: "a0fff6d4e71cc564821cd7f017543c87"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.231276 5757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:14.237526 5757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:14.237927 5757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:14.238080 5757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:14.238333 5757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:14.238811 5757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:14.238863 5757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.238895 5757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:14.238919 5757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.246822 5757 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:40555
I20260926 04:03:14.246888 5884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:40555 every 8 connection(s)
I20260926 04:03:14.247267 5757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:14.251606 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.248117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:43383 (local address 127.0.200.190:38585)
0926 04:03:14.248449 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.248451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.248549 (+ 98us) server_negotiation.cc:407] Connection header received
0926 04:03:14.249404 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.249406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.249413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.249515 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.251301 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.251303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.251303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.251310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.251335 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.251346 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.251405 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.251534 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.251548 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260926 04:03:14.251886 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.248126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:38585 (local address 127.0.200.131:43383)
0926 04:03:14.248323 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.248366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.249226 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.249608 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.249613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.249626 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.251079 (+ 1453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.251080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.251126 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.251169 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.251415 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.251423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.251427 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.251563 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.251638 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260926 04:03:14.252452 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5757
I20260926 04:03:14.252563 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:14.252758 802 external_mini_cluster.cc:1428] Setting key 6adb43ac2de83c21670d01d4497522b7
I20260926 04:03:14.252724 5885 heartbeater.cc:347] Connected to a master server at 127.0.200.190:38585
I20260926 04:03:14.252837 5885 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:14.253033 5885 heartbeater.cc:511] Master 127.0.200.190:38585 requested a full tablet report, sending...
I20260926 04:03:14.253429 5434 ts_manager.cc:194] Registered new tserver with Master: 3359d3679fad479e8bc9933e48de870b (127.0.200.131:40555)
I20260926 04:03:14.253742 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:03:14.256587 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.255366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.0.200.190:38585)
0926 04:03:14.255410 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.255412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.255423 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:14.255503 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.255505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.255510 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.255560 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.256333 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.256335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.256335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.256340 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.256356 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.256360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.256430 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.256484 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.256500 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:14.258488 5434 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:14.260339 5434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:14.264547 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.262703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:42435 (local address 127.0.0.1:41940)
0926 04:03:14.263061 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.263071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.263128 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.263295 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.263296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.263299 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.264068 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.264068 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.264086 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.264111 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.264358 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.264364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.264367 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.264435 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.264451 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":71,"threads_started":1}
I20260926 04:03:14.264547 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.262701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:40555 (local address 127.0.0.1:39068)
0926 04:03:14.262827 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.262872 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.262931 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.263253 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.263260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.263268 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.264143 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.264144 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.264153 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.264167 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.264442 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.264443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.264443 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.264470 (+ 27us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.264479 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":109,"threads_started":1}
I20260926 04:03:14.264719 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.262924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.0.200.131:40555)
0926 04:03:14.263088 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.263094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.263105 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:14.263133 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.263136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.263146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.263499 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.264246 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.264293 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.264294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.264298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.264333 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.264359 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.264582 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.264606 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.264622 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260926 04:03:14.264928 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.262806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.0.200.130:42435)
0926 04:03:14.263000 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.263005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.263126 (+ 121us) server_negotiation.cc:407] Connection header received
0926 04:03:14.263192 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.263195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.263204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.263500 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.264182 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.264192 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.264192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.264197 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.264237 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.264268 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.264361 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.264828 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.264847 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260926 04:03:14.265537 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.262942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43597 (local address 127.0.0.1:40376)
0926 04:03:14.263458 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.263468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.263514 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.264009 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.264010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.264012 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.265168 (+ 1156us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.265169 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.265175 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.265186 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.265421 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.265422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.265423 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.265465 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.265478 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":81,"threads_started":1}
I20260926 04:03:14.265403 5819 tablet_service.cc:1467] Processing CreateTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b (DEFAULT_TABLE table=test-table [id=13723a0670be4700ba6ec99c4f7f84a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:14.265611 5688 tablet_service.cc:1467] Processing CreateTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b (DEFAULT_TABLE table=test-table [id=13723a0670be4700ba6ec99c4f7f84a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:14.265764 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.263172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.0.200.129:43597)
0926 04:03:14.263333 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.263337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.263615 (+ 278us) server_negotiation.cc:407] Connection header received
0926 04:03:14.263645 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.263647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.263656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.263726 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.265271 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.265278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.265278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.265282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.265319 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.265345 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.265397 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.265645 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.265662 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260926 04:03:14.265793 5819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.265991 5688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.268466 5899 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Bootstrap starting.
I20260926 04:03:14.269165 5899 tablet_bootstrap.cc:654] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.269495 5899 log.cc:824] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.269860 5900 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Bootstrap starting.
I20260926 04:03:14.270359 5899 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: No bootstrap required, opened a new log
I20260926 04:03:14.270438 5899 ts_tablet_manager.cc:1397] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:14.270615 5900 tablet_bootstrap.cc:654] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.270938 5900 log.cc:824] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.270885 5557 tablet_service.cc:1467] Processing CreateTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b (DEFAULT_TABLE table=test-table [id=13723a0670be4700ba6ec99c4f7f84a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:14.271257 5557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.271705 5900 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: No bootstrap required, opened a new log
I20260926 04:03:14.271776 5900 ts_tablet_manager.cc:1397] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:14.272919 5899 raft_consensus.cc:348] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.273083 5899 raft_consensus.cc:374] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.273125 5899 raft_consensus.cc:729] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3359d3679fad479e8bc9933e48de870b, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.273262 5899 consensus_queue.cc:260] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.273561 5899 ts_tablet_manager.cc:1428] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 04:03:14.273567 5885 heartbeater.cc:503] Master 127.0.200.190:38585 was elected leader, sending a full tablet report...
I20260926 04:03:14.274370 5900 raft_consensus.cc:348] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.274533 5900 raft_consensus.cc:374] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.274610 5900 raft_consensus.cc:729] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879ac73abe894813ac59dcfce161f814, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.274745 5904 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: Bootstrap starting.
I20260926 04:03:14.274761 5900 consensus_queue.cc:260] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.275074 5754 heartbeater.cc:503] Master 127.0.200.190:38585 was elected leader, sending a full tablet report...
I20260926 04:03:14.275024 5900 ts_tablet_manager.cc:1428] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 04:03:14.275447 5904 tablet_bootstrap.cc:654] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.275775 5904 log.cc:824] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.276578 5904 tablet_bootstrap.cc:492] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: No bootstrap required, opened a new log
I20260926 04:03:14.276652 5904 ts_tablet_manager.cc:1397] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:14.278386 5904 raft_consensus.cc:348] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.278537 5904 raft_consensus.cc:374] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.278604 5904 raft_consensus.cc:729] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59eda72dfad433ba35d435951ac829d, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.278728 5904 consensus_queue.cc:260] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.279009 5904 ts_tablet_manager.cc:1428] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 04:03:14.279031 5623 heartbeater.cc:503] Master 127.0.200.190:38585 was elected leader, sending a full tablet report...
I20260926 04:03:14.280961 5907 raft_consensus.cc:482] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:14.281078 5907 raft_consensus.cc:504] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.281407 5907 leader_election.cc:288] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 879ac73abe894813ac59dcfce161f814 (127.0.200.130:42435), 3359d3679fad479e8bc9933e48de870b (127.0.200.131:40555)
I20260926 04:03:14.282907 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.281791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:50953 (local address 127.0.200.131:40555)
0926 04:03:14.281833 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.281835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.281842 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:03:14.281864 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.281865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.281871 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.281961 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.282688 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.282689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.282690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.282694 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.282709 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.282714 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.282772 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.282817 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.282832 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:03:14.282925 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.281549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:40555 (local address 127.0.200.129:50953)
0926 04:03:14.281721 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.281731 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.281768 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.281986 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.281987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.281990 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.282523 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.282524 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.282531 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.282541 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.282770 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.282771 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.282772 (+ 1us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.282818 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.282832 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":10,"thread_start_us":102,"threads_started":1}
I20260926 04:03:14.283098 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.281781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55625 (local address 127.0.200.130:42435)
0926 04:03:14.281831 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.281833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.281843 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:14.281868 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.281869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.281873 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.281983 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.282876 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.282878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.282878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.282884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.282899 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.282903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.282964 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.283019 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.283032 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:03:14.283099 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.281542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:42435 (local address 127.0.200.129:55625)
0926 04:03:14.281590 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.281601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.281637 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.282037 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.282038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.282041 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:14.282762 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:14.282762 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:14.282775 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:14.282787 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:14.282963 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:14.282964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:14.282964 (+ 0us) client_negotiation.cc:769] Sending connection context
0926 04:03:14.283017 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 04:03:14.283031 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 04:03:14.283546 5708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b51677e73aa4b1eafb1b54c0b0dd55b" candidate_uuid: "b59eda72dfad433ba35d435951ac829d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879ac73abe894813ac59dcfce161f814" is_pre_election: true
I20260926 04:03:14.283591 5839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b51677e73aa4b1eafb1b54c0b0dd55b" candidate_uuid: "b59eda72dfad433ba35d435951ac829d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3359d3679fad479e8bc9933e48de870b" is_pre_election: true
I20260926 04:03:14.283700 5708 raft_consensus.cc:2454] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b59eda72dfad433ba35d435951ac829d in term 0.
I20260926 04:03:14.283700 5839 raft_consensus.cc:2454] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b59eda72dfad433ba35d435951ac829d in term 0.
I20260926 04:03:14.283913 5512 leader_election.cc:302] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3359d3679fad479e8bc9933e48de870b, b59eda72dfad433ba35d435951ac829d; no voters:
I20260926 04:03:14.284099 5907 raft_consensus.cc:2781] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:14.284147 5907 raft_consensus.cc:482] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:14.284166 5907 raft_consensus.cc:3037] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.284842 5907 raft_consensus.cc:504] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.285053 5907 leader_election.cc:288] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [CANDIDATE]: Term 1 election: Requested vote from peers 879ac73abe894813ac59dcfce161f814 (127.0.200.130:42435), 3359d3679fad479e8bc9933e48de870b (127.0.200.131:40555)
I20260926 04:03:14.285111 5839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b51677e73aa4b1eafb1b54c0b0dd55b" candidate_uuid: "b59eda72dfad433ba35d435951ac829d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3359d3679fad479e8bc9933e48de870b"
I20260926 04:03:14.285140 5708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b51677e73aa4b1eafb1b54c0b0dd55b" candidate_uuid: "b59eda72dfad433ba35d435951ac829d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879ac73abe894813ac59dcfce161f814"
I20260926 04:03:14.285187 5839 raft_consensus.cc:3037] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.285203 5708 raft_consensus.cc:3037] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.285817 5708 raft_consensus.cc:2454] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b59eda72dfad433ba35d435951ac829d in term 1.
I20260926 04:03:14.285894 5839 raft_consensus.cc:2454] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b59eda72dfad433ba35d435951ac829d in term 1.
I20260926 04:03:14.286036 5511 leader_election.cc:302] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 879ac73abe894813ac59dcfce161f814, b59eda72dfad433ba35d435951ac829d; no voters:
I20260926 04:03:14.286120 5907 raft_consensus.cc:2781] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:14.286247 5907 raft_consensus.cc:686] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 1 LEADER]: Becoming Leader. State: Replica: b59eda72dfad433ba35d435951ac829d, State: Running, Role: LEADER
I20260926 04:03:14.286316 5907 consensus_queue.cc:237] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } }
I20260926 04:03:14.287216 5435 catalog_manager.cc:5495] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d reported cstate change: term changed from 0 to 1, leader changed from <none> to b59eda72dfad433ba35d435951ac829d (127.0.200.129). New cstate: current_term: 1 leader_uuid: "b59eda72dfad433ba35d435951ac829d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b59eda72dfad433ba35d435951ac829d" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:14.291986 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.291000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.0.200.129:43597)
0926 04:03:14.291058 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.291060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.291069 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:14.291108 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.291108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.291119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.291174 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.291795 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.291796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.291797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.291801 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.291814 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.291819 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.291880 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.291916 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.291927 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 04:03:14.293658 5839 raft_consensus.cc:1261] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 1 FOLLOWER]: Refusing update from remote peer b59eda72dfad433ba35d435951ac829d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:14.293697 5708 raft_consensus.cc:1261] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 1 FOLLOWER]: Refusing update from remote peer b59eda72dfad433ba35d435951ac829d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:14.294013 5907 consensus_queue.cc:1035] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [LEADER]: Connected to new peer: Peer: permanent_uuid: "879ac73abe894813ac59dcfce161f814" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:14.294152 5909 consensus_queue.cc:1035] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3359d3679fad479e8bc9933e48de870b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:14.296722 5912 mvcc.cc:205] Tried to move back new op lower bound from 7333459535024279552 to 7333459534997303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333459535024279552}]
I20260926 04:03:14.296828 5914 mvcc.cc:205] Tried to move back new op lower bound from 7333459535024279552 to 7333459534997303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333459535024279552}]
I20260926 04:03:14.296926 5913 mvcc.cc:205] Tried to move back new op lower bound from 7333459535024279552 to 7333459534997303296. Current Snapshot: MvccSnapshot[applied={T|T < 7333459535024279552}]
I20260926 04:03:14.299890 5435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:14.300011 5435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:14.300814 5435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bfedd3ef91441d39129bd1d39c99271: Sending DeleteTablet for 3 replicas of tablet 8b51677e73aa4b1eafb1b54c0b0dd55b
I20260926 04:03:14.301146 5688 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:14 UTC) from {username='slave'} at 127.0.0.1:41940
I20260926 04:03:14.301180 5557 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:14 UTC) from {username='slave'} at 127.0.0.1:40376
I20260926 04:03:14.301381 5922 tablet_replica.cc:330] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: stopping tablet replica
I20260926 04:03:14.301487 5922 raft_consensus.cc:2229] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:14.301481 5819 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:14 UTC) from {username='slave'} at 127.0.0.1:39068
I20260926 04:03:14.301597 5922 raft_consensus.cc:2258] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:14.301510 5923 tablet_replica.cc:330] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d: stopping tablet replica
I20260926 04:03:14.301762 5923 raft_consensus.cc:2229] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:14.301949 5923 raft_consensus.cc:2258] T 8b51677e73aa4b1eafb1b54c0b0dd55b P b59eda72dfad433ba35d435951ac829d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:14.302057 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5495
I20260926 04:03:14.303081 5922 ts_tablet_manager.cc:1905] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:14.305128 5922 ts_tablet_manager.cc:1918] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:14.305333 5922 log.cc:1196] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/8b51677e73aa4b1eafb1b54c0b0dd55b
I20260926 04:03:14.305584 5922 ts_tablet_manager.cc:1939] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 879ac73abe894813ac59dcfce161f814: Deleting consensus metadata
I20260926 04:03:14.306169 5420 catalog_manager.cc:4841] TS 879ac73abe894813ac59dcfce161f814 (127.0.200.130:42435): tablet 8b51677e73aa4b1eafb1b54c0b0dd55b (table test-table [id=13723a0670be4700ba6ec99c4f7f84a8]) successfully deleted
I20260926 04:03:14.306686 5924 tablet_replica.cc:330] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: stopping tablet replica
I20260926 04:03:14.306849 5924 raft_consensus.cc:2229] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:14.306974 5924 raft_consensus.cc:2258] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:14.307456 5924 ts_tablet_manager.cc:1905] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 04:03:14.308591 5420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:43597 (error 108)
I20260926 04:03:14.308669 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5626
I20260926 04:03:14.309368 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.308987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43597 (local address 127.0.0.1:40402)
0926 04:03:14.309234 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.309325 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:43597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227}
I20260926 04:03:14.309379 5924 ts_tablet_manager.cc:1918] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:14.309444 5924 log.cc:1196] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/8b51677e73aa4b1eafb1b54c0b0dd55b
W20260926 04:03:14.309509 5420 catalog_manager.cc:4584] TS b59eda72dfad433ba35d435951ac829d (127.0.200.129:43597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b51677e73aa4b1eafb1b54c0b0dd55b: Network error: Client connection negotiation failed: client connection to 127.0.200.129:43597: connect: Connection refused (error 111)
I20260926 04:03:14.309710 5924 ts_tablet_manager.cc:1939] T 8b51677e73aa4b1eafb1b54c0b0dd55b P 3359d3679fad479e8bc9933e48de870b: Deleting consensus metadata
I20260926 04:03:14.310139 5421 catalog_manager.cc:4841] TS 3359d3679fad479e8bc9933e48de870b (127.0.200.131:40555): tablet 8b51677e73aa4b1eafb1b54c0b0dd55b (table test-table [id=13723a0670be4700ba6ec99c4f7f84a8]) successfully deleted
I20260926 04:03:14.314002 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5757
I20260926 04:03:14.318413 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5404
2026-09-26T04:03:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (663 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 04:03:14.338330 802 mini_oidc.cc:152] Starting JWKS server
I20260926 04:03:14.339525 802 webserver.cc:460] Webserver started at https://127.0.0.1:39113/ using document root <none> and password file <none>
I20260926 04:03:14.339656 802 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 04:03:14.339850 802 webserver.cc:460] Webserver started at http://127.0.0.1:45239/ using document root <none> and password file <none>
2026-09-26T04:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:14Z Disabled control of system clock
I20260926 04:03:14.347673 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44003
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:42009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:44003
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39113/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/testchainca.cert with env {}
W20260926 04:03:14.429076 5931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:14.429301 5931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:14.429338 5931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:14.430516 5931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:14.430578 5931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:14.430593 5931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:14.430605 5931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:14.430617 5931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 04:03:14.430630 5931 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 04:03:14.430641 5931 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39113/jwks.json
I20260926 04:03:14.432227 5931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:42009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:44003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39113/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:14.432466 5931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:14.432636 5931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:03:14.437791 5931 server_base.cc:1034] running on GCE node
W20260926 04:03:14.437723 5937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.437723 5939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.437819 5936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.438156 5931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:14.438414 5931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:14.439582 5931 hybrid_clock.cc:648] HybridClock initialized: now 1790395394439556 us; error 41 us; skew 500 ppm
I20260926 04:03:14.440943 5931 webserver.cc:460] Webserver started at http://127.0.200.190:36385/ using document root <none> and password file <none>
I20260926 04:03:14.441207 5931 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:14.441255 5931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:14.441397 5931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:14.442515 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "2d293e64ac5f4055bf069e2014cb4dda"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "dcc4e72358b7c4d51d7f7d6a454389d0"
server_key_iv: "9bb6d4d0b9debd5e90e236751cb2b067"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.442955 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "2d293e64ac5f4055bf069e2014cb4dda"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "dcc4e72358b7c4d51d7f7d6a454389d0"
server_key_iv: "9bb6d4d0b9debd5e90e236751cb2b067"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.444470 5931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:14.445304 5945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.445544 5931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:14.445665 5931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "2d293e64ac5f4055bf069e2014cb4dda"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "dcc4e72358b7c4d51d7f7d6a454389d0"
server_key_iv: "9bb6d4d0b9debd5e90e236751cb2b067"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.445744 5931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:14.470584 5931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:14.470955 5931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:14.471133 5931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:14.475256 5931 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:44003
I20260926 04:03:14.475314 5997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:44003 every 8 connection(s)
I20260926 04:03:14.475701 5931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:14.476678 5998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.478986 5998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda: Bootstrap starting.
I20260926 04:03:14.479530 5998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.479771 5998 log.cc:824] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.480504 5998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda: No bootstrap required, opened a new log
I20260926 04:03:14.482107 5998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } }
I20260926 04:03:14.482265 5998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.482293 5998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d293e64ac5f4055bf069e2014cb4dda, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.482537 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 5931
I20260926 04:03:14.482499 5998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } }
I20260926 04:03:14.482622 5998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:14.482656 5998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:14.482666 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:14.482707 5998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.482883 802 external_mini_cluster.cc:1428] Setting key f6eecd09729deeff375557406f69a3fa
I20260926 04:03:14.483405 5998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } }
I20260926 04:03:14.483516 5998 leader_election.cc:302] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d293e64ac5f4055bf069e2014cb4dda; no voters:
I20260926 04:03:14.483717 5998 leader_election.cc:288] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:14.483752 6003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:14.483878 6003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [term 1 LEADER]: Becoming Leader. State: Replica: 2d293e64ac5f4055bf069e2014cb4dda, State: Running, Role: LEADER
I20260926 04:03:14.483961 6003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } }
I20260926 04:03:14.484015 5998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:14.484467 6004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d293e64ac5f4055bf069e2014cb4dda. Latest consensus state: current_term: 1 leader_uuid: "2d293e64ac5f4055bf069e2014cb4dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } } }
I20260926 04:03:14.484727 6004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.484663 6005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d293e64ac5f4055bf069e2014cb4dda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d293e64ac5f4055bf069e2014cb4dda" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44003 } } }
I20260926 04:03:14.485062 6005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.485428 6012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:14.486080 6012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:14.488209 6012 catalog_manager.cc:1347] Generated new cluster ID: 42c0cf2db5964bf88ed6bd2f6a053252
I20260926 04:03:14.488296 6012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:14.489276 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.483407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.0.200.190:44003)
0926 04:03:14.483706 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.483714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.483739 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:03:14.483881 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.483890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.483970 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.484363 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.484475 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.485936 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.487014 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.487622 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.487755 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.488547 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.488551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.488551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.488764 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.488810 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.488825 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.488905 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.488988 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.489067 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
I20260926 04:03:14.490275 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:03:14.498711 6012 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:14.499450 6012 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:14.503428 6012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d293e64ac5f4055bf069e2014cb4dda: Generated new TSK 0
I20260926 04:03:14.503626 6012 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 04:03:14.529847 6028 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 04:03:14.529965 6028 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 04:03:14.530207 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.514324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.0.200.190:44003)
0926 04:03:14.514400 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.514403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.514631 (+ 228us) server_negotiation.cc:407] Connection header received
0926 04:03:14.514704 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.514705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.514717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.514784 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0926 04:03:14.514998 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.515555 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.516724 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.516922 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.516987 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.517244 (+ 257us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 04:03:14.529968 (+ 12724us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 04:03:14.530141 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52096: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260926 04:03:14.530267 6002 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52096: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 04:03:14.530625 6027 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:14.514253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44003 (local address 127.0.0.1:52096)
0926 04:03:14.514485 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.514497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.514612 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.514815 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.514816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.514818 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 04:03:14.514895 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.514897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.515645 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.515646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.516616 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.516619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.517003 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.517005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.517071 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.517097 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 04:03:14.530161 (+ 13064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 04:03:14.530277 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.200.190:44003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.200.190:44003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 04:03:14.531021 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 5931
2026-09-26T04:03:14Z chronyd exiting
I20260926 04:03:14.567487 802 test_util.cc:175] -----------------------------------------------
I20260926 04:03:14.567575 802 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790395371830964-802-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (230 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 04:03:14.568333 802 mini_oidc.cc:152] Starting JWKS server
I20260926 04:03:14.569023 802 webserver.cc:460] Webserver started at https://127.0.0.1:40151/ using document root <none> and password file <none>
I20260926 04:03:14.569063 802 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 04:03:14.569208 802 webserver.cc:460] Webserver started at http://127.0.0.1:40273/ using document root <none> and password file <none>
2026-09-26T04:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:14Z Disabled control of system clock
I20260926 04:03:14.577262 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44741
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:44741
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40151/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/testchainca.cert with env {}
W20260926 04:03:14.659565 6034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:14.659833 6034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:14.659881 6034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:14.661108 6034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:14.661185 6034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:14.661206 6034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:14.661229 6034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:14.661258 6034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 04:03:14.661283 6034 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 04:03:14.661306 6034 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40151/jwks.json
I20260926 04:03:14.662933 6034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:44741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40151/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:14.663244 6034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:14.663468 6034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:14.668442 6042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.668501 6039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.668442 6040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.668681 6034 server_base.cc:1034] running on GCE node
I20260926 04:03:14.668982 6034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:14.669304 6034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:14.670472 6034 hybrid_clock.cc:648] HybridClock initialized: now 1790395394670453 us; error 51 us; skew 500 ppm
I20260926 04:03:14.671918 6034 webserver.cc:460] Webserver started at http://127.0.200.190:39629/ using document root <none> and password file <none>
I20260926 04:03:14.672178 6034 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:14.672252 6034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:14.672395 6034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:14.673506 6034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "d0bfe06aaf204827b26549eb5d9b4060"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "9428bfe351d1b6f7856060cfe77193d4"
server_key_iv: "2df5ed4ed6b45cfcb7b2f1f7d0420f29"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.673893 6034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "d0bfe06aaf204827b26549eb5d9b4060"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "9428bfe351d1b6f7856060cfe77193d4"
server_key_iv: "2df5ed4ed6b45cfcb7b2f1f7d0420f29"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.675366 6034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:14.676318 6048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.676546 6034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:14.676669 6034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "d0bfe06aaf204827b26549eb5d9b4060"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "9428bfe351d1b6f7856060cfe77193d4"
server_key_iv: "2df5ed4ed6b45cfcb7b2f1f7d0420f29"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.676760 6034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:14.692613 6034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:14.692987 6034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:14.693161 6034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:14.697680 6034 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:44741
I20260926 04:03:14.697736 6100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:44741 every 8 connection(s)
I20260926 04:03:14.698125 6034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:14.699206 6101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.701777 6101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060: Bootstrap starting.
I20260926 04:03:14.701951 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6034
I20260926 04:03:14.702052 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:14.702235 802 external_mini_cluster.cc:1428] Setting key be0295c97bfb9cddaf4a4ae5cd5bb9fe
I20260926 04:03:14.702558 6101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.702924 6101 log.cc:824] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.703737 6101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060: No bootstrap required, opened a new log
I20260926 04:03:14.705572 6101 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } }
I20260926 04:03:14.705734 6101 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.705792 6101 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bfe06aaf204827b26549eb5d9b4060, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.705914 6101 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } }
I20260926 04:03:14.705991 6101 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:14.706012 6101 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:14.706081 6101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.706838 6101 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } }
I20260926 04:03:14.706950 6101 leader_election.cc:302] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bfe06aaf204827b26549eb5d9b4060; no voters:
I20260926 04:03:14.707111 6101 leader_election.cc:288] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:14.707168 6106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:14.707316 6106 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [term 1 LEADER]: Becoming Leader. State: Replica: d0bfe06aaf204827b26549eb5d9b4060, State: Running, Role: LEADER
I20260926 04:03:14.707422 6101 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:14.707454 6106 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } }
I20260926 04:03:14.707772 6106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bfe06aaf204827b26549eb5d9b4060. Latest consensus state: current_term: 1 leader_uuid: "d0bfe06aaf204827b26549eb5d9b4060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } } }
I20260926 04:03:14.707779 6108 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0bfe06aaf204827b26549eb5d9b4060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfe06aaf204827b26549eb5d9b4060" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44741 } } }
I20260926 04:03:14.707852 6106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.707885 6108 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.708081 6112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:14.708362 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.702763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.0.200.190:44741)
0926 04:03:14.703012 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.703023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.703055 (+ 32us) server_negotiation.cc:407] Connection header received
0926 04:03:14.703620 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.703626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.703686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.703832 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.704059 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.704952 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.705753 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.706346 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.706487 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.707310 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.707314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.707314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.707569 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.707621 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.707637 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.707711 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.708086 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.708171 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260926 04:03:14.708570 6112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:14.709406 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:03:14.710322 6112 catalog_manager.cc:1347] Generated new cluster ID: 728612bd58984d6b90dbfb2a36012bc9
I20260926 04:03:14.710374 6112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:14.715430 6112 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:14.716070 6112 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:14.724906 6112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bfe06aaf204827b26549eb5d9b4060: Generated new TSK 0
I20260926 04:03:14.725148 6112 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 04:03:14.748708 6131 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 04:03:14.748910 6131 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 04:03:14.749198 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.732172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49258 (local address 127.0.200.190:44741)
0926 04:03:14.732239 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.732241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.732250 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:14.732297 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.732298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.732310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.732397 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0926 04:03:14.732583 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.733370 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.734388 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.734734 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.734810 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.734976 (+ 166us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 04:03:14.748913 (+ 13937us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 04:03:14.749114 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 04:03:14.749266 6104 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49258: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 04:03:14.749276 6130 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:14.731933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44741 (local address 127.0.0.1:49258)
0926 04:03:14.732066 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.732077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.732134 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.732443 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.732444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.732446 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 04:03:14.732505 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.732507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.733504 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.733506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.734193 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.734195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.734821 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.734822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.734868 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.734875 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 04:03:14.749090 (+ 14215us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 04:03:14.749183 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.200.190:44741: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":91,"threads_started":1}
I20260926 04:03:14.749786 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6034
2026-09-26T04:03:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 04:03:14.787696 802 mini_oidc.cc:152] Starting JWKS server
I20260926 04:03:14.788414 802 webserver.cc:460] Webserver started at https://127.0.0.1:34111/ using document root <none> and password file <none>
I20260926 04:03:14.788487 802 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 04:03:14.788681 802 webserver.cc:460] Webserver started at http://127.0.0.1:33635/ using document root <none> and password file <none>
2026-09-26T04:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:14Z Disabled control of system clock
I20260926 04:03:14.796593 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39339
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39339
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34111/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 04:03:14.881417 6137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:14.881681 6137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:14.881708 6137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:14.883080 6137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:14.883138 6137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:14.883152 6137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:14.883165 6137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:14.883179 6137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 04:03:14.883193 6137 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 04:03:14.883204 6137 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34111/jwks.json
I20260926 04:03:14.884796 6137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39339
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34111/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:14.885087 6137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:14.885413 6137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:14.890465 6143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.890756 6137 server_base.cc:1034] running on GCE node
W20260926 04:03:14.890465 6142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:14.890668 6145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:14.891121 6137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:14.891402 6137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:14.892653 6137 hybrid_clock.cc:648] HybridClock initialized: now 1790395394892622 us; error 47 us; skew 500 ppm
I20260926 04:03:14.894445 6137 webserver.cc:460] Webserver started at http://127.0.200.190:45467/ using document root <none> and password file <none>
I20260926 04:03:14.894801 6137 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:14.894889 6137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:14.895064 6137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:14.896294 6137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "09d65e18d50a4586a34ee5c65b5e15ba"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "1704d94e0d75e80fc203925ce29454d8"
server_key_iv: "7b9d2fb18471f7a62db7623f86808e4c"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.896733 6137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "09d65e18d50a4586a34ee5c65b5e15ba"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "1704d94e0d75e80fc203925ce29454d8"
server_key_iv: "7b9d2fb18471f7a62db7623f86808e4c"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.898389 6137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260926 04:03:14.899362 6151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:14.899770 6137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:14.899917 6137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "09d65e18d50a4586a34ee5c65b5e15ba"
format_stamp: "Formatted at 2026-09-26 04:03:14 on dist-test-slave-j8c1"
server_key: "1704d94e0d75e80fc203925ce29454d8"
server_key_iv: "7b9d2fb18471f7a62db7623f86808e4c"
server_key_version: "encryptionkey@0"
I20260926 04:03:14.900049 6137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:14.920883 6137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:14.921248 6137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:14.921555 6137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:14.925671 6137 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:39339
I20260926 04:03:14.925753 6203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:39339 every 8 connection(s)
I20260926 04:03:14.926162 6137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:14.926997 6204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:14.930151 6204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba: Bootstrap starting.
I20260926 04:03:14.930876 6204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:14.931138 6204 log.cc:824] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:14.931489 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6137
I20260926 04:03:14.931628 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:14.931821 802 external_mini_cluster.cc:1428] Setting key 3d2ef364275fc225e829b876c8be7ef2
I20260926 04:03:14.931948 6204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba: No bootstrap required, opened a new log
I20260926 04:03:14.933982 6204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } }
I20260926 04:03:14.934170 6204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:14.934221 6204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d65e18d50a4586a34ee5c65b5e15ba, State: Initialized, Role: FOLLOWER
I20260926 04:03:14.934360 6204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } }
I20260926 04:03:14.934458 6204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:14.934500 6204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:14.934582 6204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:14.935200 6204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } }
I20260926 04:03:14.935328 6204 leader_election.cc:302] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09d65e18d50a4586a34ee5c65b5e15ba; no voters:
I20260926 04:03:14.935530 6204 leader_election.cc:288] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:14.935599 6209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:14.935745 6209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [term 1 LEADER]: Becoming Leader. State: Replica: 09d65e18d50a4586a34ee5c65b5e15ba, State: Running, Role: LEADER
I20260926 04:03:14.935811 6204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:14.935876 6209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } }
I20260926 04:03:14.936215 6210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } } }
I20260926 04:03:14.936298 6210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.936201 6209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d65e18d50a4586a34ee5c65b5e15ba. Latest consensus state: current_term: 1 leader_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d65e18d50a4586a34ee5c65b5e15ba" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39339 } } }
I20260926 04:03:14.936475 6209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:14.936589 6216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:14.937071 6216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:14.938724 6216 catalog_manager.cc:1347] Generated new cluster ID: 317bb3e58702458ca423cadff2a2d5f8
I20260926 04:03:14.938869 6216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:14.939082 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.932450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.0.200.190:39339)
0926 04:03:14.932682 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.932691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.932722 (+ 31us) server_negotiation.cc:407] Connection header received
0926 04:03:14.933266 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.933272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.933331 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.933476 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:14.933698 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.935280 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.936623 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.937219 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.937346 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.938240 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:14.938245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:14.938245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:14.938472 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:14.938516 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:14.938533 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:14.938618 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:14.938722 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 04:03:14.938804 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260926 04:03:14.939997 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:03:14.944620 6216 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:14.945328 6216 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:14.951678 6216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09d65e18d50a4586a34ee5c65b5e15ba: Generated new TSK 0
I20260926 04:03:14.951903 6216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:14.961445 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:14.957522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44194 (local address 127.0.200.190:39339)
0926 04:03:14.957576 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:14.957578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:14.957635 (+ 57us) server_negotiation.cc:407] Connection header received
0926 04:03:14.957732 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:14.957733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:14.957747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:14.957852 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0926 04:03:14.958033 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.958693 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.959639 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.959875 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.959970 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.960235 (+ 265us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 04:03:14.961171 (+ 936us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 04:03:14.961338 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260926 04:03:14.961527 6208 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 04:03:14.961498 6233 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:14.957279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39339 (local address 127.0.0.1:44194)
0926 04:03:14.957536 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:14.957552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:14.957657 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:14.957851 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:14.957852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:14.957854 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 04:03:14.957939 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.957943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.958801 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.958803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.959490 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:14.959492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:14.959998 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:14.960000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:14.960071 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:14.960077 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 04:03:14.961290 (+ 1213us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 04:03:14.961389 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.200.190:39339: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34111/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":195,"threads_started":1}
I20260926 04:03:14.962181 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6137
E20260926 04:03:14.972219 6234 webserver.cc:533] Webserver: sslize failed:
E20260926 04:03:14.972307 6234 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T04:03:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T04:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:15Z Disabled control of system clock
I20260926 04:03:15.010223 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36903
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36903
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/testchainca.cert with env {}
W20260926 04:03:15.095057 6238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:15.095291 6238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:15.095317 6238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:15.096623 6238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:15.096675 6238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:15.096694 6238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:15.096707 6238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:15.096720 6238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 04:03:15.096776 6238 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 04:03:15.096793 6238 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 04:03:15.098389 6238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:15.098697 6238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:15.098949 6238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:15.104689 6243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:15.104694 6246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:15.104822 6238 server_base.cc:1034] running on GCE node
W20260926 04:03:15.104694 6244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:15.105304 6238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:15.105654 6238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:15.106827 6238 hybrid_clock.cc:648] HybridClock initialized: now 1790395395106816 us; error 53 us; skew 500 ppm
I20260926 04:03:15.108124 6238 webserver.cc:460] Webserver started at http://127.0.200.190:40421/ using document root <none> and password file <none>
I20260926 04:03:15.108379 6238 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:15.108451 6238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:15.108630 6238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:15.109834 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "3c00fc158de0424c9d2baaa705080c2d"
format_stamp: "Formatted at 2026-09-26 04:03:15 on dist-test-slave-j8c1"
server_key: "e1730f20d0bca533cec10f82a04644d2"
server_key_iv: "1e82a978e67412b5ff880b9b0d20e424"
server_key_version: "encryptionkey@0"
I20260926 04:03:15.110232 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "3c00fc158de0424c9d2baaa705080c2d"
format_stamp: "Formatted at 2026-09-26 04:03:15 on dist-test-slave-j8c1"
server_key: "e1730f20d0bca533cec10f82a04644d2"
server_key_iv: "1e82a978e67412b5ff880b9b0d20e424"
server_key_version: "encryptionkey@0"
I20260926 04:03:15.111721 6238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:15.112679 6252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:15.112932 6238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:15.113049 6238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "3c00fc158de0424c9d2baaa705080c2d"
format_stamp: "Formatted at 2026-09-26 04:03:15 on dist-test-slave-j8c1"
server_key: "e1730f20d0bca533cec10f82a04644d2"
server_key_iv: "1e82a978e67412b5ff880b9b0d20e424"
server_key_version: "encryptionkey@0"
I20260926 04:03:15.113140 6238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:15.135669 6238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:15.136051 6238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:15.136225 6238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:15.140499 6238 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:36903
I20260926 04:03:15.140535 6304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:36903 every 8 connection(s)
I20260926 04:03:15.140897 6238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:15.141940 6305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:15.144517 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d: Bootstrap starting.
I20260926 04:03:15.145143 6305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:15.145298 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6238
I20260926 04:03:15.145386 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:15.145519 6305 log.cc:824] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:15.145581 802 external_mini_cluster.cc:1428] Setting key cb59250afa968f19e4eb25a88a6c6ef8
I20260926 04:03:15.146379 6305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d: No bootstrap required, opened a new log
I20260926 04:03:15.148940 6305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } }
I20260926 04:03:15.149124 6305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:15.149158 6305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c00fc158de0424c9d2baaa705080c2d, State: Initialized, Role: FOLLOWER
I20260926 04:03:15.149284 6305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } }
I20260926 04:03:15.149372 6305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:15.149411 6305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:15.149466 6305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:15.150228 6305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } }
I20260926 04:03:15.150338 6305 leader_election.cc:302] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c00fc158de0424c9d2baaa705080c2d; no voters:
I20260926 04:03:15.150527 6305 leader_election.cc:288] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:15.150646 6310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:15.150807 6310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [term 1 LEADER]: Becoming Leader. State: Replica: 3c00fc158de0424c9d2baaa705080c2d, State: Running, Role: LEADER
I20260926 04:03:15.150835 6305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:15.150913 6310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } }
I20260926 04:03:15.151331 6311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c00fc158de0424c9d2baaa705080c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } } }
I20260926 04:03:15.151342 6310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c00fc158de0424c9d2baaa705080c2d. Latest consensus state: current_term: 1 leader_uuid: "3c00fc158de0424c9d2baaa705080c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c00fc158de0424c9d2baaa705080c2d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36903 } } }
I20260926 04:03:15.151525 6310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:15.151445 6311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:15.151793 6319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:15.151883 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:15.146120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.0.200.190:36903)
0926 04:03:15.146337 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:15.146344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:15.146372 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:03:15.146909 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:15.146916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:15.146977 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:15.147091 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:15.147304 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.148154 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.149317 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.149938 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.150071 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:15.150861 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:15.150865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:15.150865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:15.151074 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:15.151117 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:15.151132 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:15.151175 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:15.151319 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 04:03:15.151428 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260926 04:03:15.152635 6319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:15.152853 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:03:15.154368 6319 catalog_manager.cc:1347] Generated new cluster ID: 8f8e9506c2c8459bab3e2d29a761bb54
I20260926 04:03:15.154422 6319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:15.169257 6319 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:15.169824 6319 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:15.181354 6319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c00fc158de0424c9d2baaa705080c2d: Generated new TSK 0
I20260926 04:03:15.181568 6319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:15.273598 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:15.192586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.0.200.190:36903)
0926 04:03:15.192630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:15.192632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:15.192666 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:03:15.192757 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:15.192758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:15.192768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:15.192829 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0926 04:03:15.192991 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.193722 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.194976 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.195242 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.195290 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:15.195559 (+ 269us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 04:03:15.273310 (+ 77751us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 04:03:15.273518 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60822: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 04:03:15.273684 6309 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60822: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 04:03:15.273795 6334 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:15.192461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36903 (local address 127.0.0.1:60822)
0926 04:03:15.192605 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:15.192616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:15.192686 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:15.192850 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:15.192851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:15.192853 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0926 04:03:15.192922 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:15.192925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.193854 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.193855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:15.194866 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:15.194870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:15.195383 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:15.195384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:15.195436 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:15.195442 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 04:03:15.273510 (+ 78068us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 04:03:15.273641 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.200.190:36903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260926 04:03:15.274140 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6238
2026-09-26T04:03:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (291 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:15 dist-test-slave-j8c1 krb5kdc[6346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:15 dist-test-slave-j8c1 krb5kdc[6346](info): set up 2 sockets
Sep 26 04:03:15 dist-test-slave-j8c1 krb5kdc[6346](info): commencing operation
krb5kdc: starting...
W20260926 04:03:16.510859 802 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:16 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:16.634668 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46207
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:46207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:16.715821 6359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:16.716112 6359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:16.716156 6359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:16.717507 6359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:16.717602 6359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:16.717626 6359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:16.717645 6359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:16.717674 6359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:16.719748 6359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:46207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:16.720095 6359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:16.720379 6359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:03:16.725826 6359 server_base.cc:1034] running on GCE node
W20260926 04:03:16.725751 6364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:16.725744 6365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:16.725751 6367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:16.726344 6359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:16.726761 6359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:16.727936 6359 hybrid_clock.cc:648] HybridClock initialized: now 1790395396727924 us; error 36 us; skew 500 ppm
Sep 26 04:03:16 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:16.731180 6359 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:16.731971 6359 webserver.cc:460] Webserver started at http://127.0.200.190:33179/ using document root <none> and password file <none>
I20260926 04:03:16.732257 6359 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:16.732333 6359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:16.732475 6359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:16.733927 6359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "84f7eb8408b5410dbb4c8038f3b19ea9"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "f3689287350b8ae2d1780b74c7d62083"
server_key_iv: "32d89990f30917907ff0c4cf93a8d5f9"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.734344 6359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "84f7eb8408b5410dbb4c8038f3b19ea9"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "f3689287350b8ae2d1780b74c7d62083"
server_key_iv: "32d89990f30917907ff0c4cf93a8d5f9"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.735898 6359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:16.736835 6374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:16.737089 6359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:16.737238 6359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "84f7eb8408b5410dbb4c8038f3b19ea9"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "f3689287350b8ae2d1780b74c7d62083"
server_key_iv: "32d89990f30917907ff0c4cf93a8d5f9"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.737351 6359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:16.750362 6359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:16.752628 6359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:16.752825 6359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:16.757293 6359 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:46207
I20260926 04:03:16.757376 6426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:46207 every 8 connection(s)
I20260926 04:03:16.757934 6359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:16.758698 6427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:16.759784 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6359
I20260926 04:03:16.759904 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:16.760100 802 external_mini_cluster.cc:1428] Setting key d942b8ad1f21a0c8fb52215eedfc0aa9
I20260926 04:03:16.761445 6427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9: Bootstrap starting.
I20260926 04:03:16.762074 6427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:16.762444 6427 log.cc:824] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:16.763180 6427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9: No bootstrap required, opened a new log
I20260926 04:03:16.765040 6427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } }
I20260926 04:03:16.765214 6427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:16.765273 6427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f7eb8408b5410dbb4c8038f3b19ea9, State: Initialized, Role: FOLLOWER
I20260926 04:03:16.765435 6427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } }
I20260926 04:03:16.765534 6427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:16.765579 6427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:16.765627 6427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:16.766201 6427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } }
I20260926 04:03:16.766326 6427 leader_election.cc:302] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84f7eb8408b5410dbb4c8038f3b19ea9; no voters:
I20260926 04:03:16.766577 6432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:16.766537 6427 leader_election.cc:288] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:16.766733 6432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [term 1 LEADER]: Becoming Leader. State: Replica: 84f7eb8408b5410dbb4c8038f3b19ea9, State: Running, Role: LEADER
I20260926 04:03:16.766820 6432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } }
I20260926 04:03:16.766899 6427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 04:03:16 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:16.767143 6432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } } }
I20260926 04:03:16.767297 6432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:16.767158 6433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f7eb8408b5410dbb4c8038f3b19ea9. Latest consensus state: current_term: 1 leader_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7eb8408b5410dbb4c8038f3b19ea9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46207 } } }
I20260926 04:03:16.767431 6433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:16.767622 6437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:16.768167 6437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:16.770982 6437 catalog_manager.cc:1347] Generated new cluster ID: e8649ec74b61446a8dcb80e901d861ec
I20260926 04:03:16.771041 6437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:16.771634 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:16.760692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.0.200.190:46207)
0926 04:03:16.760892 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:16.760900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:16.760928 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:03:16.761325 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:16.761331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:16.761377 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:16.761529 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:16.762223 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.763226 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.764006 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.764450 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.764580 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:16.767632 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:16.767636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:16.767637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:16.767650 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:16.769772 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:16.770193 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:16.770196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:16.770197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:16.770304 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:16.770595 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:16.770596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:16.770596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:16.770906 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:16.771043 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:16.771297 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 04:03:16.771434 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260926 04:03:16.782963 6437 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:16.783527 6437 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:16.793499 6437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84f7eb8408b5410dbb4c8038f3b19ea9: Generated new TSK 0
I20260926 04:03:16.793812 6437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:16.821780 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:16.907881 6453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:16.908118 6453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:16.908144 6453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:16.909439 6453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:16.909495 6453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:16.909525 6453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:16.911146 6453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:16.911439 6453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:16.911705 6453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:16.917117 6461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:16.917197 6453 server_base.cc:1034] running on GCE node
W20260926 04:03:16.917117 6458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:16.917171 6459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:16.917840 6453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:16.918128 6453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:16.919332 6453 hybrid_clock.cc:648] HybridClock initialized: now 1790395396919314 us; error 31 us; skew 500 ppm
Sep 26 04:03:16 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:16.922159 6453 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:16.922797 6453 webserver.cc:460] Webserver started at http://127.0.200.129:42321/ using document root <none> and password file <none>
I20260926 04:03:16.923059 6453 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:16.923130 6453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:16.923285 6453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:16.924415 6453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "81f1492c4b2c4bacb079094aae682ef3"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "0a8c4057b016bcbeb1aeecb81a70a3e6"
server_key_iv: "a906ceed0df44b001e1c467e16c1f6c7"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.924808 6453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "81f1492c4b2c4bacb079094aae682ef3"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "0a8c4057b016bcbeb1aeecb81a70a3e6"
server_key_iv: "a906ceed0df44b001e1c467e16c1f6c7"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.926301 6453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:16.927374 6468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:16.927630 6453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:16.927785 6453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "81f1492c4b2c4bacb079094aae682ef3"
format_stamp: "Formatted at 2026-09-26 04:03:16 on dist-test-slave-j8c1"
server_key: "0a8c4057b016bcbeb1aeecb81a70a3e6"
server_key_iv: "a906ceed0df44b001e1c467e16c1f6c7"
server_key_version: "encryptionkey@0"
I20260926 04:03:16.927876 6453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:16.944293 6453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:16.946486 6453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:16.946724 6453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:16.947017 6453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:16.947355 6453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:16.947414 6453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:16.947456 6453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:16.947501 6453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:16.953596 6453 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:41677
I20260926 04:03:16.953644 6581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:41677 every 8 connection(s)
I20260926 04:03:16.954008 6453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:16.957347 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6453
I20260926 04:03:16.957477 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:16.957683 802 external_mini_cluster.cc:1428] Setting key 20a66a7d9a3c96949b84c692305a89cc
Sep 26 04:03:16 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:16.965085 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:16.954834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:43309 (local address 127.0.200.190:46207)
0926 04:03:16.954917 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:16.954919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:16.955087 (+ 168us) server_negotiation.cc:407] Connection header received
0926 04:03:16.956153 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:16.956154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:16.956164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:16.956221 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:16.956896 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.957421 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.958803 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.959028 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.959115 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:16.961770 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:16.961772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:16.961773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:16.961781 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:16.963597 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:16.964031 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:16.964032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:16.964032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:16.964097 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:16.964364 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:16.964365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:16.964366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:16.964515 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:16.964590 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:16.964859 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 04:03:16.964958 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 04:03:16.965127 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:16.954789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46207 (local address 127.0.200.129:43309)
0926 04:03:16.954952 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:16.954992 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:16.956043 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:16.956283 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:16.956286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:16.956581 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:16.956802 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:16.956807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.957587 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.957588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:16.958697 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:16.958700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:16.959127 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:16.959128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:16.959265 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:16.959894 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:16.959906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:16.961662 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:16.963700 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:16.963707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:16.963709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:16.963949 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:16.964187 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:16.964187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:16.964188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:16.964291 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:16.964586 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:16.964591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:16.964720 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 04:03:16.964811 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 04:03:16.964953 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260926 04:03:16.965831 6582 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46207
I20260926 04:03:16.965957 6582 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:16.966169 6582 heartbeater.cc:511] Master 127.0.200.190:46207 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:03:16.966881 6391 ts_manager.cc:194] Registered new tserver with Master: 81f1492c4b2c4bacb079094aae682ef3 (127.0.200.129:41677)
I20260926 04:03:16.967654 6391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:43309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:16.987214 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:17.069311 6587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:17.069562 6587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:17.069602 6587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:17.070811 6587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:17.070883 6587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:17.070931 6587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:17.072453 6587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:17.072731 6587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:17.073002 6587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:17.078176 6595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:17.078184 6593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:17.078190 6592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:17.078604 6587 server_base.cc:1034] running on GCE node
I20260926 04:03:17.078863 6587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:17.079144 6587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:17.080334 6587 hybrid_clock.cc:648] HybridClock initialized: now 1790395397080298 us; error 52 us; skew 500 ppm
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.083130 6587 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:17.083617 6587 webserver.cc:460] Webserver started at http://127.0.200.130:33711/ using document root <none> and password file <none>
I20260926 04:03:17.083875 6587 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:17.083928 6587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:17.084084 6587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:17.085206 6587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "76f1ddb2be02492b999875d3b0c425e5"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "8747474fa0bac146c861de55c31d8f50"
server_key_iv: "fafc0b4d572d29c9b3f4a451e09ca257"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.085601 6587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "76f1ddb2be02492b999875d3b0c425e5"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "8747474fa0bac146c861de55c31d8f50"
server_key_iv: "fafc0b4d572d29c9b3f4a451e09ca257"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.087061 6587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:17.087975 6602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.088222 6587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:17.088326 6587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "76f1ddb2be02492b999875d3b0c425e5"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "8747474fa0bac146c861de55c31d8f50"
server_key_iv: "fafc0b4d572d29c9b3f4a451e09ca257"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.088415 6587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:17.113231 6587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:17.115703 6587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:17.115901 6587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:17.116184 6587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:17.116545 6587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:17.116605 6587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.116669 6587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:17.116698 6587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.123494 6587 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:35659
I20260926 04:03:17.123539 6715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:35659 every 8 connection(s)
I20260926 04:03:17.123909 6587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:17.132997 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6587
I20260926 04:03:17.133116 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:17.133330 802 external_mini_cluster.cc:1428] Setting key ad6d6d658a90eb6ce24bf47fe937a57a
I20260926 04:03:17.135527 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.124783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:49417 (local address 127.0.200.190:46207)
0926 04:03:17.124846 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.124847 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.125102 (+ 255us) server_negotiation.cc:407] Connection header received
0926 04:03:17.126106 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.126107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.126118 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.126184 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.126945 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.127465 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.128630 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.128823 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.128885 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.131835 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:17.131837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:17.131837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:17.131844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:17.133736 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.134369 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.134371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.134371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.134441 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.134790 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.134791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.134792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.134924 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:17.135014 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:17.135328 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 04:03:17.135426 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:03:17.135744 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.124737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46207 (local address 127.0.200.130:49417)
0926 04:03:17.124913 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:17.124960 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:17.125993 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:17.126241 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:17.126244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:17.126524 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:17.126832 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:17.126837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.127569 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.127570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:17.128505 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:17.128507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.128926 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.128927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:17.129035 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.129675 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:17.129685 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:17.131720 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:17.133831 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:17.133839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:17.133842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:17.134248 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:17.134530 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:17.134531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:17.134531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:17.134709 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:17.135002 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:17.135009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:17.135174 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 04:03:17.135290 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 04:03:17.135470 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260926 04:03:17.136499 6716 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46207
I20260926 04:03:17.136593 6716 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:17.136777 6716 heartbeater.cc:511] Master 127.0.200.190:46207 requested a full tablet report, sending...
I20260926 04:03:17.137260 6391 ts_manager.cc:194] Registered new tserver with Master: 76f1ddb2be02492b999875d3b0c425e5 (127.0.200.130:35659)
I20260926 04:03:17.137679 6391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:49417
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:17.162179 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:17.246923 6721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:17.247161 6721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:17.247203 6721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:17.248413 6721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:17.248485 6721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:17.248531 6721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:17.250042 6721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:17.250325 6721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:17.250617 6721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:17.256027 6729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:17.256006 6726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:17.256156 6721 server_base.cc:1034] running on GCE node
W20260926 04:03:17.256031 6727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:17.256474 6721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:17.256805 6721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:17.257999 6721 hybrid_clock.cc:648] HybridClock initialized: now 1790395397257988 us; error 28 us; skew 500 ppm
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.261117 6721 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:17.261603 6721 webserver.cc:460] Webserver started at http://127.0.200.131:39613/ using document root <none> and password file <none>
I20260926 04:03:17.261857 6721 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:17.261924 6721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:17.262072 6721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:17.263227 6721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "970e77f5d88b431cb74225d733070603"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "d2f4b3a74dd75be93044a00b78ffcb3e"
server_key_iv: "648271b1adb420b3ef9bc90c96754d9e"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.263578 6721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "970e77f5d88b431cb74225d733070603"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "d2f4b3a74dd75be93044a00b78ffcb3e"
server_key_iv: "648271b1adb420b3ef9bc90c96754d9e"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.264941 6721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:17.265686 6736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.265996 6721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:17.266106 6721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "970e77f5d88b431cb74225d733070603"
format_stamp: "Formatted at 2026-09-26 04:03:17 on dist-test-slave-j8c1"
server_key: "d2f4b3a74dd75be93044a00b78ffcb3e"
server_key_iv: "648271b1adb420b3ef9bc90c96754d9e"
server_key_version: "encryptionkey@0"
I20260926 04:03:17.266203 6721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:17.281198 6721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:17.283812 6721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:17.284046 6721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:17.284298 6721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:17.284614 6721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:17.284670 6721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.284714 6721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:17.284752 6721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:17.290925 6721 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:46321
I20260926 04:03:17.290962 6849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:46321 every 8 connection(s)
I20260926 04:03:17.291384 6721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:17.297611 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6721
I20260926 04:03:17.297706 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:17.297927 802 external_mini_cluster.cc:1428] Setting key f8de998d67fd71c31a6e8a2152d5e114
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:17.301460 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.292207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:41509 (local address 127.0.200.190:46207)
0926 04:03:17.292336 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.292343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.292561 (+ 218us) server_negotiation.cc:407] Connection header received
0926 04:03:17.293593 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.293595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.293608 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.293688 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.294392 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.294986 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.296030 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.296215 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.296274 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.298763 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:17.298764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:17.298764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:17.298798 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:17.300187 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.300586 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.300587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.300587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.300638 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.300899 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.300899 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.300900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.301004 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:17.301055 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:17.301327 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 04:03:17.301394 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 04:03:17.301698 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.292101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46207 (local address 127.0.200.131:41509)
0926 04:03:17.292386 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:17.292459 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:17.293461 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:17.293780 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:17.293784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:17.294069 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:17.294309 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:17.294313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.295070 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.295071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:17.295948 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:17.295950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.296290 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.296290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:17.296387 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.296899 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:17.296908 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:17.298681 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:17.300273 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:17.300278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:17.300280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:17.300518 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:17.300716 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:17.300717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:17.300718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:17.300836 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:17.301062 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:17.301065 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:17.301198 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 04:03:17.301283 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:03:17.301428 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":157,"threads_started":1}
I20260926 04:03:17.302173 6850 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46207
I20260926 04:03:17.302271 6850 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:17.302443 6850 heartbeater.cc:511] Master 127.0.200.190:46207 requested a full tablet report, sending...
I20260926 04:03:17.302866 6391 ts_manager.cc:194] Registered new tserver with Master: 970e77f5d88b431cb74225d733070603 (127.0.200.131:46321)
I20260926 04:03:17.303248 6391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:41509
I20260926 04:03:17.309988 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/test-admin.keytab.
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.327813 802 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 04:03:17.329787 6857 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:17.328957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56404 (local address 127.0.200.129:41677)
0926 04:03:17.329130 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.329135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.329145 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:17.329179 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.329181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.329195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.329266 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.329696 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56404: BlockingRecv error: recv got EOF from 127.0.0.1:56404 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
W20260926 04:03:17.329924 802 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc' of type FILE
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.331212 802 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.332376 802 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:03:17.341770 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.333041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.0.200.129:41677)
0926 04:03:17.333084 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.333086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.333158 (+ 72us) server_negotiation.cc:407] Connection header received
0926 04:03:17.333217 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.333218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.333228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.333272 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.333698 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.334924 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.335705 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.336059 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.336119 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.338442 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:17.338450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:17.338450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:17.338457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:17.340355 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.340790 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.340793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.340794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.340856 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.341169 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.341171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.341171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.341336 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:17.341456 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:17.341618 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 04:03:17.341690 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 04:03:17.343794 6857 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:17.343181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.0.200.129:41677)
0926 04:03:17.343252 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.343259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.343267 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:17.343329 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.343329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.343339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.343384 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.343750 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56420: BlockingRecv error: recv got EOF from 127.0.0.1:56420 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 04:03:17.343884 802 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc' of type FILE
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.344914 802 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.345852 802 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:03:17.353420 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.346458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.0.200.129:41677)
0926 04:03:17.346533 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.346534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.346543 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:17.346672 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.346673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.346681 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.346730 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.347144 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.347673 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.348348 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.348557 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.348613 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.350848 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:17.350849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:17.350850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:17.350856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:17.352252 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.352659 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.352660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.352660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.352717 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.352951 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.352952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.352952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.353063 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:17.353154 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:17.353286 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 04:03:17.353352 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 04:03:17.355229 6857 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:17.354703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.0.200.129:41677)
0926 04:03:17.354773 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.354774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.354802 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:03:17.354853 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.354854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.354862 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.354903 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.355174 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56432: BlockingRecv error: recv got EOF from 127.0.0.1:56432 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 04:03:17.355270 802 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790395371830964-802-0/krb5kdc/krb5cc' of type FILE
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.356285 802 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:17.357175 802 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:03:17.364718 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:17.357791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.0.200.129:41677)
0926 04:03:17.357868 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:17.357870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:17.357906 (+ 36us) server_negotiation.cc:407] Connection header received
0926 04:03:17.357956 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:17.357957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:17.357965 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:17.358016 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:17.358395 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.358954 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.359705 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:17.359891 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:17.359955 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:17.362109 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:17.362110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:17.362111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:17.362119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:17.363567 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.363974 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.363975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.363976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.364036 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:17.364232 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:17.364234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:17.364234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:17.364337 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:17.364440 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:17.364538 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 04:03:17.364615 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 04:03:17.365408 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6453
I20260926 04:03:17.370311 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6587
I20260926 04:03:17.375162 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6721
I20260926 04:03:17.379905 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6359
2026-09-26T04:03:17Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1926 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6870](info): set up 2 sockets
Sep 26 04:03:17 dist-test-slave-j8c1 krb5kdc[6870](info): commencing operation
krb5kdc: starting...
W20260926 04:03:18.422907 802 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:18 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab.
Entry for principal oryx/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab.
I20260926 04:03:18.547078 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33111
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab
--principal=oryx/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:33111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:18.632503 6883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:18.632741 6883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:18.632776 6883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:18.634210 6883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:18.634281 6883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:18.634297 6883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:18.634310 6883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:18.634323 6883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:18.636274 6883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:33111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab
--principal=oryx/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:18.636595 6883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:18.636863 6883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:18.642091 6891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:18.642063 6889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:18.642306 6883 server_base.cc:1034] running on GCE node
W20260926 04:03:18.642107 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:18.642655 6883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:18.642927 6883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:18.644080 6883 hybrid_clock.cc:648] HybridClock initialized: now 1790395398644071 us; error 31 us; skew 500 ppm
Sep 26 04:03:18 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:18.646673 6883 init.cc:377] Logged in from keytab as oryx/127.0.200.190@KRBTEST.COM (short username oryx)
I20260926 04:03:18.647207 6883 webserver.cc:460] Webserver started at http://127.0.200.190:36099/ using document root <none> and password file <none>
I20260926 04:03:18.647475 6883 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:18.647553 6883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:18.647714 6883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:18.648814 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "45c0700ef6f440488586dac0ef1f14ee"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "ffc9fb25b8f8344491023faaef88e6c9"
server_key_iv: "f518a3316829c1e072987bff9738765a"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.649197 6883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "45c0700ef6f440488586dac0ef1f14ee"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "ffc9fb25b8f8344491023faaef88e6c9"
server_key_iv: "f518a3316829c1e072987bff9738765a"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.650678 6883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:18.651567 6898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:18.651773 6883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:18.651889 6883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "45c0700ef6f440488586dac0ef1f14ee"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "ffc9fb25b8f8344491023faaef88e6c9"
server_key_iv: "f518a3316829c1e072987bff9738765a"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.651985 6883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:18.689407 6883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:18.691619 6883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:18.691783 6883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:18.695884 6883 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:33111
I20260926 04:03:18.695926 6950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:33111 every 8 connection(s)
I20260926 04:03:18.696309 6883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:18.697301 6951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:18.699604 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Bootstrap starting.
I20260926 04:03:18.700168 6951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:18.700412 6951 log.cc:824] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:18.701344 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: No bootstrap required, opened a new log
I20260926 04:03:18.702580 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6883
I20260926 04:03:18.702716 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:18.702907 802 external_mini_cluster.cc:1428] Setting key d5e3d10f92d21e6ebb281580c5a2cce3
I20260926 04:03:18.703408 6951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } }
I20260926 04:03:18.703737 6951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:18.703841 6951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c0700ef6f440488586dac0ef1f14ee, State: Initialized, Role: FOLLOWER
I20260926 04:03:18.703975 6951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } }
I20260926 04:03:18.704078 6951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:18.704125 6951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:18.704167 6951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:18.705281 6951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } }
I20260926 04:03:18.705402 6951 leader_election.cc:302] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45c0700ef6f440488586dac0ef1f14ee; no voters:
I20260926 04:03:18.705546 6951 leader_election.cc:288] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:18.705602 6956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:18.705751 6956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 LEADER]: Becoming Leader. State: Replica: 45c0700ef6f440488586dac0ef1f14ee, State: Running, Role: LEADER
I20260926 04:03:18.705802 6951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:18.705847 6956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } }
I20260926 04:03:18.706173 6957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } }
I20260926 04:03:18.706163 6956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c0700ef6f440488586dac0ef1f14ee. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } }
I20260926 04:03:18.706255 6957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:18.706264 6956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:18.707260 6963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:18.707800 6963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:18.709761 6963 catalog_manager.cc:1347] Generated new cluster ID: c3823ba6a43f404abb464f2ff91f1170
I20260926 04:03:18.709811 6963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:18 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:18.714630 6963 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:18.715173 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:18.703482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.0.200.190:33111)
0926 04:03:18.703719 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:18.703730 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:18.703764 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:03:18.703885 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:18.703893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:18.703950 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:18.704102 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:18.704745 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.706191 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.707409 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.708115 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.708230 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:18.711055 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:18.711059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:18.711059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:18.711071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:18.713055 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:18.713493 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:18.713496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:18.713497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:18.713628 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:18.713871 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:18.713872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:18.713872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:18.714306 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:18.714495 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:18.714781 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 04:03:18.714955 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":105,"threads_started":1}
I20260926 04:03:18.715402 6963 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:18.719457 6963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Generated new TSK 0
I20260926 04:03:18.719650 6963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
I20260926 04:03:18.754801 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:18.842126 6977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:18.842341 6977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:18.842367 6977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:18.843638 6977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:18.843685 6977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:18.843714 6977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:18.845264 6977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:18.845518 6977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:18.845784 6977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:18.851615 6983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:18.851642 6985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:18.851639 6982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:18.852244 6977 server_base.cc:1034] running on GCE node
I20260926 04:03:18.852512 6977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:18.852794 6977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:18.853998 6977 hybrid_clock.cc:648] HybridClock initialized: now 1790395398853985 us; error 29 us; skew 500 ppm
Sep 26 04:03:18 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:18.857368 6977 init.cc:377] Logged in from keytab as oryx/127.0.200.129@KRBTEST.COM (short username oryx)
I20260926 04:03:18.857873 6977 webserver.cc:460] Webserver started at http://127.0.200.129:35505/ using document root <none> and password file <none>
I20260926 04:03:18.858132 6977 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:18.858203 6977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:18.858373 6977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:18.859549 6977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "b2b520e255af42ec9730fa5b161178b8"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "5f7d76dd9f66c3c0632c2e15fcdf23ff"
server_key_iv: "c76d421e2c7b9fd6fe7488993960c85e"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.859958 6977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "b2b520e255af42ec9730fa5b161178b8"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "5f7d76dd9f66c3c0632c2e15fcdf23ff"
server_key_iv: "c76d421e2c7b9fd6fe7488993960c85e"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.861433 6977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:18.862389 6992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:18.862721 6977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:18.862826 6977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b2b520e255af42ec9730fa5b161178b8"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "5f7d76dd9f66c3c0632c2e15fcdf23ff"
server_key_iv: "c76d421e2c7b9fd6fe7488993960c85e"
server_key_version: "encryptionkey@0"
I20260926 04:03:18.862890 6977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:18.871335 6977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:18.873660 6977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:18.873785 6977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:18.873986 6977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:18.874310 6977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:18.874346 6977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:18.874370 6977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:18.874418 6977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:18.880241 6977 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:45461
I20260926 04:03:18.880314 7105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:45461 every 8 connection(s)
I20260926 04:03:18.880659 6977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:18 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:18.890210 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 6977
I20260926 04:03:18.890224 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:18.881434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:45091 (local address 127.0.200.190:33111)
0926 04:03:18.881510 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:18.881512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:18.881758 (+ 246us) server_negotiation.cc:407] Connection header received
0926 04:03:18.882253 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:18.882254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:18.882269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:18.882321 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:18.883431 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.883958 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.884964 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.885168 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.885226 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:18.887494 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:18.887496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:18.887496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:18.887507 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:18.888848 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:18.889242 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:18.889243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:18.889243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:18.889331 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:18.889588 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:18.889588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:18.889589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:18.889696 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:18.889747 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:18.889996 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 04:03:18.890112 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:03:18.890318 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:18.890486 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:18.881420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:45091)
0926 04:03:18.881629 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:18.881667 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:18.882163 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:18.882402 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:18.882406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:18.883120 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:18.883331 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:18.883335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.884057 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.884058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:18.884877 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:18.884879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:18.885249 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:18.885249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:18.885353 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:18.885907 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:18.885916 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:18.887374 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:18.888929 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:18.888936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:18.888938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:18.889172 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:18.889418 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:18.889418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:18.889419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:18.889527 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:18.889754 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:18.889758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:18.889881 (+ 123us) client_negotiation.cc:769] Sending connection context
0926 04:03:18.889968 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 04:03:18.890213 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":127,"threads_started":1}
I20260926 04:03:18.890528 802 external_mini_cluster.cc:1428] Setting key 75575cf7b54ce9ea4906043fd6f509d5
I20260926 04:03:18.891091 7106 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:18.891204 7106 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:18.891412 7106 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:18.891927 6914 ts_manager.cc:194] Registered new tserver with Master: b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461)
I20260926 04:03:18.892575 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:45091
WARNING: no policy specified for oryx/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
I20260926 04:03:18.918386 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:19.003755 7111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.004017 7111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.004060 7111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.005364 7111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.005427 7111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.005474 7111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:19.007189 7111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:19.007494 7111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:19.007764 7111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.012928 7117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.012909 7119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.012915 7116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.013278 7111 server_base.cc:1034] running on GCE node
I20260926 04:03:19.013473 7111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.013688 7111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.014855 7111 hybrid_clock.cc:648] HybridClock initialized: now 1790395399014839 us; error 28 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.017693 7111 init.cc:377] Logged in from keytab as oryx/127.0.200.130@KRBTEST.COM (short username oryx)
I20260926 04:03:19.018199 7111 webserver.cc:460] Webserver started at http://127.0.200.130:35611/ using document root <none> and password file <none>
I20260926 04:03:19.018460 7111 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.018543 7111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.018733 7111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:19.019870 7111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "ce99795d5e014de183abc4b49576e3d2"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "04d6c71abb4618bec6431b245e5dbc41"
server_key_iv: "9d34b87f0aa81a4456844f0e2f456144"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.020251 7111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "ce99795d5e014de183abc4b49576e3d2"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "04d6c71abb4618bec6431b245e5dbc41"
server_key_iv: "9d34b87f0aa81a4456844f0e2f456144"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.021736 7111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:19.022801 7126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.023171 7111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:19.023291 7111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "ce99795d5e014de183abc4b49576e3d2"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "04d6c71abb4618bec6431b245e5dbc41"
server_key_iv: "9d34b87f0aa81a4456844f0e2f456144"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.023384 7111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.033586 7111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.036187 7111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:19.036398 7111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.036655 7111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:19.036995 7111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:19.037053 7111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.037096 7111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:19.037124 7111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.043499 7111 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:39859
I20260926 04:03:19.043627 7239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:39859 every 8 connection(s)
I20260926 04:03:19.043941 7111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:19.053969 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7111
I20260926 04:03:19.054088 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:19.054342 802 external_mini_cluster.cc:1428] Setting key 2efced30916c3294ec69310e7477966b
I20260926 04:03:19.056114 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.044803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43249 (local address 127.0.200.190:33111)
0926 04:03:19.044898 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.044901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.045273 (+ 372us) server_negotiation.cc:407] Connection header received
0926 04:03:19.045743 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.045744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.045754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.045811 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.046911 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.047600 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.048730 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.048955 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.049034 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.051808 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.051810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.051811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.051818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.053672 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.054436 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.054437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.054438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.054526 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.055019 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.055021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.055021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.055247 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.055334 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.055822 (+ 488us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.055911 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 04:03:19.056208 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.044865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:43249)
0926 04:03:19.045143 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.045182 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.045659 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.045863 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.045867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.046470 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.046797 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.046804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.047711 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.047712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.048605 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.048607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.049105 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.049106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.049228 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.049857 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.049866 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.051660 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.053879 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.053887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.053889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.054333 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.054708 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.054709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.054710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.054874 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.055356 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.055363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.055567 (+ 204us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.055696 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.055901 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":145,"threads_started":1}
I20260926 04:03:19.056717 7240 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:19.056851 7240 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:19.057046 7240 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:19.057473 6914 ts_manager.cc:194] Registered new tserver with Master: ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:19.057873 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:43249
WARNING: no policy specified for oryx/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
I20260926 04:03:19.082922 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:19.165685 7245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.165920 7245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.165946 7245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.167220 7245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.167270 7245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.167326 7245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:19.168784 7245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:19.169004 7245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:19.169189 7245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.174664 7253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.174682 7251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.174817 7245 server_base.cc:1034] running on GCE node
W20260926 04:03:19.174657 7250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.175168 7245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.175467 7245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.176646 7245 hybrid_clock.cc:648] HybridClock initialized: now 1790395399176642 us; error 55 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.185379 7245 init.cc:377] Logged in from keytab as oryx/127.0.200.131@KRBTEST.COM (short username oryx)
I20260926 04:03:19.185896 7245 webserver.cc:460] Webserver started at http://127.0.200.131:45505/ using document root <none> and password file <none>
I20260926 04:03:19.186094 7245 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.186136 7245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.186228 7245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:19.187399 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "72f2ba1281ba43499bf09b79bfa073f1"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "f1f439cae4c43a8e54b8978580b05174"
server_key_iv: "3265bec17820503fb593585a69085e68"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.187731 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "72f2ba1281ba43499bf09b79bfa073f1"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "f1f439cae4c43a8e54b8978580b05174"
server_key_iv: "3265bec17820503fb593585a69085e68"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.189054 7245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:19.189756 7260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.189949 7245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:19.190068 7245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "72f2ba1281ba43499bf09b79bfa073f1"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "f1f439cae4c43a8e54b8978580b05174"
server_key_iv: "3265bec17820503fb593585a69085e68"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.190148 7245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.197096 7245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.199748 7245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:19.199921 7245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.200184 7245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:19.200563 7245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:19.200611 7245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.200649 7245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:19.200675 7245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.207858 7245 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:40151
I20260926 04:03:19.207926 7373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:40151 every 8 connection(s)
I20260926 04:03:19.208300 7245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:19.217985 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.209090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:35341 (local address 127.0.200.190:33111)
0926 04:03:19.209172 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.209174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.209465 (+ 291us) server_negotiation.cc:407] Connection header received
0926 04:03:19.209972 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.209974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.209985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.210058 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.211106 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.211595 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.212603 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.212788 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.212849 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.215139 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.215141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.215142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.215149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.216457 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.216944 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.216946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.216946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.217013 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.217273 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.217274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.217275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.217406 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.217482 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.217778 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.217892 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 04:03:19.218250 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7245
I20260926 04:03:19.218343 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:19.218534 802 external_mini_cluster.cc:1428] Setting key dbde13e0ceee10a47e92bdafaa9a7b5e
I20260926 04:03:19.218755 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.209031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:35341)
0926 04:03:19.209341 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.209378 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.209862 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.210113 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.210116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.210769 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.210968 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.210973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.211688 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.211689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.212514 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.212516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.212864 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.212864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.212967 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.213568 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.213578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.215032 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.216579 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.216586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.216589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.216844 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.217082 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.217082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.217083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.217199 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.217483 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.217490 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.217636 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.217750 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.217930 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":96,"threads_started":1}
I20260926 04:03:19.219269 7374 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:19.219395 7374 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:19.219604 7374 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:19.220042 6914 ts_manager.cc:194] Registered new tserver with Master: 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131:40151)
I20260926 04:03:19.220436 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:35341
I20260926 04:03:19.230762 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.200.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.189.keytab.
Entry for principal oryx/127.0.200.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.189.keytab.
I20260926 04:03:19.258958 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:43611
--webserver_interface=127.0.200.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.189.keytab
--principal=oryx/127.0.200.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:33111,127.0.200.189:43611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:19.340444 7379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.340665 7379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.340688 7379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.341863 7379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:19.341910 7379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.341924 7379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.341938 7379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:19.341993 7379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:19.343640 7379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:33111,127.0.200.189:43611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.189.keytab
--principal=oryx/127.0.200.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:43611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.200.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
W20260926 04:03:19.343755 7379 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.200.190:33111,127.0.200.189:43611'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 04:03:19.343950 7379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:19.344201 7379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.349345 7385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.349350 7387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.349345 7384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.349702 7379 server_base.cc:1034] running on GCE node
I20260926 04:03:19.349969 7379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.350294 7379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.351460 7379 hybrid_clock.cc:648] HybridClock initialized: now 1790395399351446 us; error 26 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.354034 7379 init.cc:377] Logged in from keytab as oryx/127.0.200.189@KRBTEST.COM (short username oryx)
I20260926 04:03:19.354535 7379 webserver.cc:460] Webserver started at http://127.0.200.189:40839/ using document root <none> and password file <none>
I20260926 04:03:19.354894 7379 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.354964 7379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.355100 7379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:19.356256 7379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/instance:
uuid: "02076a495c44497996d0923adfd9de22"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "9f9180c0661989323ee5b40887dc5d41"
server_key_iv: "a47176dd456ee7a520ab65fed3179790"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.356622 7379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal/instance:
uuid: "02076a495c44497996d0923adfd9de22"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "9f9180c0661989323ee5b40887dc5d41"
server_key_iv: "a47176dd456ee7a520ab65fed3179790"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.357975 7379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:19.358738 7394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.358920 7379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:19.359026 7379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "02076a495c44497996d0923adfd9de22"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "9f9180c0661989323ee5b40887dc5d41"
server_key_iv: "a47176dd456ee7a520ab65fed3179790"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.359113 7379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.377784 7379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.379761 7379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:19.379935 7379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.383942 7379 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.189:43611
I20260926 04:03:19.383976 7446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.189:43611 every 8 connection(s)
I20260926 04:03:19.384315 7379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
I20260926 04:03:19.385269 7447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:19.386998 7447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.189@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:19.394127 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7379
I20260926 04:03:19.396198 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.387749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.0.200.190:33111)
0926 04:03:19.387823 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.387826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.388030 (+ 204us) server_negotiation.cc:407] Connection header received
0926 04:03:19.388453 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.388453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.388462 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.388512 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.389134 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.389617 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.390800 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.391003 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.391101 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.393296 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.393297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.393298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.393306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.394724 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.395134 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.395136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.395136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.395200 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.395466 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.395467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.395468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.395629 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.395713 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.396015 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.396106 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 04:03:19.396405 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.387736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36002)
0926 04:03:19.387912 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.387946 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.388373 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.388570 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.388573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.388827 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.389057 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.389063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.389703 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.389704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.390704 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.390707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.391109 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.391110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.391248 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.391736 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.391752 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.393191 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.394816 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.394821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.394824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.395052 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.395274 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.395275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.395276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.395390 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.395703 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.395709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.395857 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.395992 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.396156 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260926 04:03:19.397032 7447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.189@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
I20260926 04:03:19.403293 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.394706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.0.200.189:43611)
0926 04:03:19.394874 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.394879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.394944 (+ 65us) server_negotiation.cc:407] Connection header received
0926 04:03:19.395006 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.395009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.395062 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.395151 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.395729 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.396598 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.397515 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.397836 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.397916 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.400279 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.400286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.400287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.400293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.401966 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.402331 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.402335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.402335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.402398 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.402628 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.402629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.402629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.402776 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.402934 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.403005 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.403119 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260926 04:03:19.404294 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.397307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:49990)
0926 04:03:19.397339 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.397348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.397383 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.397618 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.397618 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.397761 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.397800 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.397801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.398438 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.398439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.399002 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.399004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.399388 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.399388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.399452 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.399849 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.399855 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.401029 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.403222 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.403223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.403223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.403440 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.403660 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.403660 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.403661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.403716 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.403958 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.403959 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.403994 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.404088 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.404181 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 04:03:19.404399 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.397356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.0.200.189:43611)
0926 04:03:19.397489 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.397490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.397495 (+ 5us) server_negotiation.cc:407] Connection header received
0926 04:03:19.397507 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.397508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.397514 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.397613 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.397884 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.398360 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.399102 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.399305 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.399386 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.401128 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.401129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.401130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.401136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.403107 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.403510 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.403511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.403512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.403576 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.403785 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.403786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.403787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.403894 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.403967 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.404109 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.404232 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260926 04:03:19.405735 7447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Bootstrap starting.
I20260926 04:03:19.406409 7447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:19.406766 7447 log.cc:824] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:19.407514 7447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: No bootstrap required, opened a new log
I20260926 04:03:19.408895 7447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } }
I20260926 04:03:19.409039 7447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:19.409077 7447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02076a495c44497996d0923adfd9de22, State: Initialized, Role: FOLLOWER
I20260926 04:03:19.409217 7447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } }
I20260926 04:03:19.409480 7454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } } }
I20260926 04:03:19.409551 7447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:19.409569 7454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 04:03:19.411275 7465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:03:19.411365 7465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:19.412954 7379 master_runner.cc:428] Detected that this master 02076a495c44497996d0923adfd9de22 is joining an existing cluster
I20260926 04:03:19.413013 7379 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.200.189:43611
I20260926 04:03:19.420974 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.413984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.0.200.190:33111)
0926 04:03:19.414062 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.414064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.414201 (+ 137us) server_negotiation.cc:407] Connection header received
0926 04:03:19.414285 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.414286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.414297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.414353 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.414803 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.415590 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.416383 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.416595 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.416679 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.417841 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.417843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.417843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.417851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.419591 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420034 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.420035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.420035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.420094 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420325 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.420326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.420326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.420480 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.420538 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.420764 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.420869 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:19.421046 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.414196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.0.200.189:43611)
0926 04:03:19.414231 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.414232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.414303 (+ 71us) server_negotiation.cc:407] Connection header received
0926 04:03:19.414385 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.414386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.414394 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.414420 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.414880 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.415581 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.416696 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.416985 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.417025 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.418088 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.418089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.418089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.418096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.419821 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420214 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.420215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.420215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.420262 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420439 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.420440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.420440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.420564 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.420638 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.420783 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.420928 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":21}
I20260926 04:03:19.420989 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.413923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36010)
0926 04:03:19.414136 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.414147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.414227 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.414376 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.414377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.414664 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.414722 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.414723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.415674 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.415675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.416296 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.416298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.416680 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.416681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.416743 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.417222 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.417227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.417741 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.419691 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.419692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.419693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.419943 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.420167 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.420168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.420235 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.420596 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.420598 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.420650 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.420729 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.420866 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":46,"thread_start_us":76,"threads_started":1}
I20260926 04:03:19.421222 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.414142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:49994)
0926 04:03:19.414246 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.414255 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.414325 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.414473 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.414474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.414745 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.414804 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.414806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.415668 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.415669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.416600 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.416602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.417104 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.417105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.417174 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.417545 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.417550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.417992 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.419892 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.419893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.419894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.420163 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.420320 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.420321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.420322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.420372 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.420631 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.420631 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.420681 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.420810 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.421058 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":180,"thread_start_us":52,"threads_started":1}
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.190@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
I20260926 04:03:19.431674 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.423885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.0.200.189:43611)
0926 04:03:19.423928 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.423930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.424117 (+ 187us) server_negotiation.cc:407] Connection header received
0926 04:03:19.424203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.424204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.424212 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.424257 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.424776 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.425221 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.426384 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.426630 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.426728 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.428913 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.428915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.428915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.428922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.430374 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.430852 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.430854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.430854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.430912 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.431115 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.431116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.431116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.431226 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.431295 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.431492 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.431571 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:03:19.431685 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.423835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:50002)
0926 04:03:19.423988 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.424039 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.424128 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.424265 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.424269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.424626 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.424694 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.424696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.425304 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.425305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.426290 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.426293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.426741 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.426742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.426845 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.427343 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.427349 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.428815 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.430477 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.430485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.430488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.430766 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.430979 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.430980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.430980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.431046 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.431299 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.431306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.431359 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.431451 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.431578 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260926 04:03:19.432256 6914 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.200.189:43611: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "02076a495c44497996d0923adfd9de22"
member_type: NON_VOTER
last_known_addr {
host: "127.0.200.189"
port: 43611
}
attrs {
promote: true
}
}
dest_uuid: "45c0700ef6f440488586dac0ef1f14ee"
cas_config_opid_index: -1
I20260926 04:03:19.432566 6914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } }
I20260926 04:03:19.433020 7475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 02076a495c44497996d0923adfd9de22 (127.0.200.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } }
I20260926 04:03:19.433462 7421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:19.434185 7475 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.200.189:43611
I20260926 04:03:19.434402 7379 master.cc:537] Master@127.0.200.189:43611 shutting down...
I20260926 04:03:19.434446 7421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 04:03:19.434790 7475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:19.434401 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } }
I20260926 04:03:19.435343 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
W20260926 04:03:19.435698 6901 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 04:03:19.436913 6901 consensus_peers.cc:487] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee -> Peer 02076a495c44497996d0923adfd9de22 (127.0.200.189:43611): Couldn't send request to peer 02076a495c44497996d0923adfd9de22. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 04:03:19.437305 7379 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:19.437398 7379 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:19.437440 7379 tablet_replica.cc:330] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: stopping tablet replica
I20260926 04:03:19.450258 7379 master.cc:552] Master@127.0.200.189:43611 shutdown complete.
I20260926 04:03:19.450338 7379 master_runner.cc:305] Clearing existing system tablet
I20260926 04:03:19.450470 7379 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.450538 7379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.451262 7379 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:19.451787 7479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.451905 7379 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.451961 7379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:19.451983 7379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "02076a495c44497996d0923adfd9de22"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "9f9180c0661989323ee5b40887dc5d41"
server_key_iv: "a47176dd456ee7a520ab65fed3179790"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.453055 7379 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:19.454598 7379 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 04:03:19.454689 7379 log.cc:1196] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 04:03:19.454900 7379 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Deleting consensus metadata
I20260926 04:03:19.455052 7379 master_runner.cc:315] Copying system tablet from 127.0.200.190:33111
I20260926 04:03:19.455638 7379 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: tablet copy: Beginning tablet copy session from remote peer at address 127.0.200.190:33111
I20260926 04:03:19.462349 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.455925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36012)
0926 04:03:19.456063 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.456074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.456259 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.456410 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.456411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.456574 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.456633 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.456634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.457583 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.457585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.458188 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.458190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.458687 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.458688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.458786 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.459307 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.459313 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.459625 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.461293 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.461294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.461294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.461457 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.461689 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.461690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.461690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.461768 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.462056 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.462057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.462103 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.462166 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.462260 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260926 04:03:19.462390 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.455992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36012 (local address 127.0.200.190:33111)
0926 04:03:19.456056 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.456059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.456259 (+ 200us) server_negotiation.cc:407] Connection header received
0926 04:03:19.456325 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.456326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.456339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.456398 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.456703 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.457489 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.458275 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.458602 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.458678 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.459713 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.459714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.459714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.459722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.461203 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.461526 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.461528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.461528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.461592 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.461842 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.461843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.461844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.461981 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.462052 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.462211 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.462290 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:19.463219 6935 tablet_copy_service.cc:140] P 45c0700ef6f440488586dac0ef1f14ee: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 02076a495c44497996d0923adfd9de22 ({username='oryx', principal='oryx/127.0.200.189@KRBTEST.COM'} at 127.0.0.1:36012)
I20260926 04:03:19.463375 6935 tablet_copy_service.cc:161] P 45c0700ef6f440488586dac0ef1f14ee: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 02076a495c44497996d0923adfd9de22 at {username='oryx', principal='oryx/127.0.200.189@KRBTEST.COM'} at 127.0.0.1:36012: session id = 02076a495c44497996d0923adfd9de22-00000000000000000000000000000000
I20260926 04:03:19.464131 6935 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 04:03:19.464793 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 04:03:19.465912 7379 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: tablet copy: Starting download of 0 data blocks...
I20260926 04:03:19.466069 7379 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: tablet copy: Starting download of 1 WAL segments...
I20260926 04:03:19.467131 7379 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 04:03:19.467989 6935 tablet_copy_service.cc:342] P 45c0700ef6f440488586dac0ef1f14ee: Request end of tablet copy session 02076a495c44497996d0923adfd9de22-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.200.189@KRBTEST.COM'} at 127.0.0.1:36012
I20260926 04:03:19.468052 6935 tablet_copy_service.cc:433] P 45c0700ef6f440488586dac0ef1f14ee: ending tablet copy session 02076a495c44497996d0923adfd9de22-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 02076a495c44497996d0923adfd9de22
I20260926 04:03:19.470482 7379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.471899 7486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.471899 7487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.471920 7489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.472103 7379 server_base.cc:1034] running on GCE node
I20260926 04:03:19.472190 7379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.472347 7379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.473496 7379 hybrid_clock.cc:648] HybridClock initialized: now 1790395399473487 us; error 25 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.475039 7379 init.cc:377] Logged in from keytab as oryx/127.0.200.189@KRBTEST.COM (short username oryx)
I20260926 04:03:19.475345 7379 webserver.cc:460] Webserver started at http://127.0.200.189:35439/ using document root <none> and password file <none>
I20260926 04:03:19.475504 7379 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.475551 7379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.476231 7379 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.476879 7496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.477021 7379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.477066 7379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "02076a495c44497996d0923adfd9de22"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "9f9180c0661989323ee5b40887dc5d41"
server_key_iv: "a47176dd456ee7a520ab65fed3179790"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.477130 7379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.495548 7379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.497342 7379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.500574 7548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.189:43611 every 8 connection(s)
I20260926 04:03:19.500559 7379 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.189:43611
I20260926 04:03:19.500919 7379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
I20260926 04:03:19.501811 7549 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:03:19.502413 7549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Bootstrap starting.
I20260926 04:03:19.505635 7549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:03:19.505899 7549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: Bootstrap complete.
I20260926 04:03:19.506114 7549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } }
I20260926 04:03:19.506208 7549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 02076a495c44497996d0923adfd9de22, State: Initialized, Role: LEARNER
I20260926 04:03:19.506254 7549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } }
I20260926 04:03:19.506426 7549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:19.506428 7553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } }
I20260926 04:03:19.506736 7553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: LEARNER
I20260926 04:03:19.508172 7564 catalog_manager.cc:1259] Loaded cluster ID: c3823ba6a43f404abb464f2ff91f1170
I20260926 04:03:19.508219 7564 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: loading cluster ID for follower catalog manager: success
I20260926 04:03:19.509421 7564 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: acquiring CA information for follower catalog manager: success
I20260926 04:03:19.509722 7564 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:03:19.512729 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.504981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.0.200.189:43611)
0926 04:03:19.505153 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.505154 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.505163 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:19.505191 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.505192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.505201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.505282 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.506041 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.507166 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.508114 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.508403 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.508475 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.509960 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.509961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.509962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.509968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.511568 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.511972 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.511973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.511973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.512024 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.512231 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.512232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.512232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.512327 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.512420 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.512546 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.512619 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for oryx/127.0.200.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.188.keytab.
Entry for principal oryx/127.0.200.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.188.keytab.
I20260926 04:03:19.538048 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:45769
--webserver_interface=127.0.200.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.188.keytab
--principal=oryx/127.0.200.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:19.623383 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.623598 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.623620 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.624792 7567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:19.624833 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.624848 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.624861 7567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:19.624872 7567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:19.626430 7567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.188.keytab
--principal=oryx/127.0.200.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.200.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:19.626704 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:19.626924 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.631896 7575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.631915 7573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.631906 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.631999 7567 server_base.cc:1034] running on GCE node
I20260926 04:03:19.632423 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.632779 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.633924 7567 hybrid_clock.cc:648] HybridClock initialized: now 1790395399633888 us; error 51 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.636576 7567 init.cc:377] Logged in from keytab as oryx/127.0.200.188@KRBTEST.COM (short username oryx)
I20260926 04:03:19.637051 7567 webserver.cc:460] Webserver started at http://127.0.200.188:38651/ using document root <none> and password file <none>
I20260926 04:03:19.637290 7567 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.637359 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.637511 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:19.638780 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/instance:
uuid: "1aef1e65b86443d795d601e47f86fbd0"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "c4bfefe1c7c8969c9f77049be2d6fc36"
server_key_iv: "ad2cde0d7d888af19a16220fc9fb35e7"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.639156 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal/instance:
uuid: "1aef1e65b86443d795d601e47f86fbd0"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "c4bfefe1c7c8969c9f77049be2d6fc36"
server_key_iv: "ad2cde0d7d888af19a16220fc9fb35e7"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.640514 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:19.641245 7582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.641415 7567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:19.641513 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "1aef1e65b86443d795d601e47f86fbd0"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "c4bfefe1c7c8969c9f77049be2d6fc36"
server_key_iv: "ad2cde0d7d888af19a16220fc9fb35e7"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.641602 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.661621 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.664135 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:19.664305 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.668437 7567 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.188:45769
I20260926 04:03:19.668500 7634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.188:45769 every 8 connection(s)
I20260926 04:03:19.668849 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
I20260926 04:03:19.669888 7635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:19.671525 7635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:03:19.673322 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7567
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.188@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
I20260926 04:03:19.682257 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.672357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36024 (local address 127.0.200.190:33111)
0926 04:03:19.672408 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.672411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.672474 (+ 63us) server_negotiation.cc:407] Connection header received
0926 04:03:19.672890 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.672891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.672899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.672949 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.673771 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.674814 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.676355 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.676685 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.676754 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.679124 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.679126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.679126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.679135 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.680695 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.681204 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.681206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.681206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.681272 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.681551 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.681552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.681553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.681702 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.681781 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.682038 (+ 257us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.682134 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:03:19.682339 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.672177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36024)
0926 04:03:19.672351 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.672385 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.672813 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.673000 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.673003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.673360 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.673671 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.673677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.674997 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.674998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.676205 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.676207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.676802 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.676803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.676904 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.677453 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.677462 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.679006 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.680779 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.680786 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.680789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.681125 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.681344 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.681345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.681345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.681478 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.681780 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.681785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.681902 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.681987 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.682143 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":21,"thread_start_us":66,"threads_started":1}
I20260926 04:03:19.682943 7635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:03:19.683779 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.673864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.0.200.188:45769)
0926 04:03:19.674025 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.674029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.674046 (+ 17us) server_negotiation.cc:407] Connection header received
0926 04:03:19.674073 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.674074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.674089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.674150 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.674981 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.676265 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.677227 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.677645 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.677719 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.680393 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.680400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.680401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.680408 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.682285 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.682738 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.682741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.682741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.682799 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.683023 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.683023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.683024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.683224 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.683397 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.683521 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.683633 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.188@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
I20260926 04:03:19.689711 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.683317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.0.200.189:43611)
0926 04:03:19.683364 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.683366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.683373 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:03:19.683397 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.683398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.683411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.683464 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.683797 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.684506 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.685174 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.685369 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.685427 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.687381 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.687383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.687383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.687391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.688577 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.688921 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.688921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.688921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.688975 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.689244 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.689245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.689246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.689371 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.689439 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.689552 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.689625 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:03:19.689723 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.683222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:50022)
0926 04:03:19.683275 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.683287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.683327 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.683480 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.683482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.683676 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.683733 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.683735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.684576 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.684577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.685088 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.685090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.685453 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.685454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.685518 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.685982 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.685987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.687290 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.688678 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.688679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.688680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.688849 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.689034 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.689035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.689035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.689091 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.689439 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.689440 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.689481 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.689535 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.689611 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 04:03:19.690207 7635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.188@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
I20260926 04:03:19.697641 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.690525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45898)
0926 04:03:19.690600 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.690611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.690657 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.690861 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.690862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.691084 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.691128 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.691131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.691914 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.691914 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.692393 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.692394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.692772 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.692773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.692814 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.693337 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.693342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.694622 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.696482 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.696483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.696483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.696695 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.696928 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.696928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.696929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.697002 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.697270 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.697272 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.697314 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.697393 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.697497 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 04:03:19.697646 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.690587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.0.200.188:45769)
0926 04:03:19.690623 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.690625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.690637 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:19.690758 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.690759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.690769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.690851 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.691216 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.691843 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.692485 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.692703 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.692759 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.694714 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.694716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.694716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.694723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.696395 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.696792 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.696793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.696793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.696847 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.697083 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.697083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.697084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.697202 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.697261 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.697422 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.697544 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":18}
I20260926 04:03:19.699723 7635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Bootstrap starting.
I20260926 04:03:19.700503 7635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:19.700784 7635 log.cc:824] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:19.701471 7635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: No bootstrap required, opened a new log
I20260926 04:03:19.702922 7635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } }
I20260926 04:03:19.703053 7635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:19.703076 7635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aef1e65b86443d795d601e47f86fbd0, State: Initialized, Role: FOLLOWER
I20260926 04:03:19.703164 7635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } }
I20260926 04:03:19.703395 7640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } } }
I20260926 04:03:19.703459 7635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:19.703506 7640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 04:03:19.705240 7651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:03:19.705305 7651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:19.707695 7567 master_runner.cc:428] Detected that this master 1aef1e65b86443d795d601e47f86fbd0 is joining an existing cluster
I20260926 04:03:19.707747 7567 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.200.188:45769
I20260926 04:03:19.716986 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.708713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36028)
0926 04:03:19.708909 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.708921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.708982 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.709162 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.709163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.709452 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.709532 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.709534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710506 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.710507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.711467 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.711469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.711998 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.712082 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.712637 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.712643 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.713436 (+ 793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.715713 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.715714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.715715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.715961 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.716235 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.716236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.716237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.716300 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.716650 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.716651 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.716710 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.716772 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.716857 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":112,"threads_started":1}
I20260926 04:03:19.717010 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.708767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.0.200.190:33111)
0926 04:03:19.708811 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.708813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.708968 (+ 155us) server_negotiation.cc:407] Connection header received
0926 04:03:19.709042 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.709043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.709062 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.709113 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.709620 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710419 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711568 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.711862 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711903 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.713544 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.713545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.713546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.713554 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.715592 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.716065 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.716067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.716068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.716139 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.716408 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.716409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.716409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.716562 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.716637 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.716819 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.716914 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395398, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.190@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
I20260926 04:03:19.735337 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6977
I20260926 04:03:19.744277 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.709184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45906)
0926 04:03:19.709300 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.709311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.709377 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.709536 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.709537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.709753 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.709832 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.709834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710735 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.710736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.711583 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.711586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.712060 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.712061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.712121 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.739340 (+ 27219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.739346 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.739940 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.742369 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.742370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.742371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.742711 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.743017 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.743017 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.743018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.743105 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.743541 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.743542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.743581 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.743670 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.744128 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260926 04:03:19.744580 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.709197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.0.200.188:45769)
0926 04:03:19.709235 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.709237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.709357 (+ 120us) server_negotiation.cc:407] Connection header received
0926 04:03:19.709441 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.709442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.709449 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.709486 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.709911 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710650 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711674 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.711953 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711992 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.740098 (+ 28106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.740099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.740100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.740106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.742246 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.742844 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.742845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.742846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.742922 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.743212 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.743213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.743213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.743408 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.743496 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.743817 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.744503 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":611,"server-negotiator.queue_time_us":17}
I20260926 04:03:19.745071 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.709097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.0.200.189:43611)
0926 04:03:19.709147 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.709149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.709346 (+ 197us) server_negotiation.cc:407] Connection header received
0926 04:03:19.709438 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.709440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.709450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.709480 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.709905 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710622 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711630 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.711919 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.711957 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.740811 (+ 28854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.740818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.740819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.740828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.742674 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.743792 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.743793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.743793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.743862 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.744080 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.744082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.744082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.744336 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.744446 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.744812 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.744923 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:03:19.745428 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.708935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:50024)
0926 04:03:19.709287 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.709297 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.709368 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.709541 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.709542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.709760 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.709817 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.709818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.710704 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.710704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.711540 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.711542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.712026 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.712027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.712082 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.739850 (+ 27768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.739855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.740694 (+ 839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.742772 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.742773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.742774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.742979 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.743939 (+ 960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.743940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.743940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.743994 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.744533 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.744534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.744618 (+ 84us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.745185 (+ 567us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.745324 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":72,"threads_started":1}
I20260926 04:03:19.746377 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.719220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45920)
0926 04:03:19.719301 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.719313 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.719356 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.719755 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.719756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.719957 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.720001 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.720002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.721039 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.721040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.721550 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.721551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.722050 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.722051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.722104 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.722524 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.722529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.724148 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.744989 (+ 20841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.744991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.744992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.745224 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.745493 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.745494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.745494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.745557 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.745955 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.745957 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.746033 (+ 76us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.746106 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.746227 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:03:19.746652 7660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.719406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.0.200.188:45769)
0926 04:03:19.719580 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.719582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.719590 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:19.719625 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.719626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.719644 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.719723 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.720098 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.720945 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.721632 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.721973 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.722044 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.724262 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.724264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.724264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.724272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.744884 (+ 20612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.745304 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.745305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.745305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.745354 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.745624 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.745624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.745625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.745854 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.745948 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.746351 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.746528 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260926 04:03:19.746686 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:45461
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:33111
--webserver_port=35505
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:03:19.747099 6914 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.200.188:45769: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0"
member_type: NON_VOTER
last_known_addr {
host: "127.0.200.188"
port: 45769
}
attrs {
promote: true
}
}
dest_uuid: "45c0700ef6f440488586dac0ef1f14ee"
cas_config_opid_index: 5
I20260926 04:03:19.747352 6914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } }
I20260926 04:03:19.748142 7475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.749244 7475 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.200.188:45769
I20260926 04:03:19.749323 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.749416 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:19.749609 7609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:19.751216 7609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 04:03:19.751487 7567 master.cc:537] Master@127.0.200.188:45769 shutting down...
I20260926 04:03:19.752604 7474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 04:03:19.754192 6902 consensus_peers.cc:487] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee -> Peer 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188:45769): Couldn't send request to peer 1aef1e65b86443d795d601e47f86fbd0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 04:03:19.755447 7567 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:19.755541 7567 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:19.755586 7567 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: stopping tablet replica
I20260926 04:03:19.759845 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.747873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.0.200.189:43611)
0926 04:03:19.747913 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.747915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.755240 (+ 7325us) server_negotiation.cc:407] Connection header received
0926 04:03:19.755270 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.755272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.755299 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.755376 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:19.755639 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.756479 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.758233 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.758965 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.759044 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.759134 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.759711 (+ 577us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.759767 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:19.759526 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.747813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:50038)
0926 04:03:19.755096 (+ 7283us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.755110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.755174 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.755457 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.755458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.755460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:19.755528 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.755531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.756711 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.756713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.758115 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.758119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.759174 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.759175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.759254 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.759285 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.759343 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.759399 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7258}
I20260926 04:03:19.760880 7523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 LEARNER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 04:03:19.761078 7474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 04:03:19.761663 7523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.763175 7553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c0700ef6f440488586dac0ef1f14ee. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.763265 7553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: LEARNER
I20260926 04:03:19.763300 7553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: NON_VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: true } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.763343 7553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: LEARNER
I20260926 04:03:19.763587 7474 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: attempting to promote NON_VOTER 02076a495c44497996d0923adfd9de22 to VOTER
I20260926 04:03:19.763805 7474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } }
I20260926 04:03:19.764389 7523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 LEARNER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 04:03:19.764580 7475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 04:03:19.765054 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.764322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45930)
0926 04:03:19.764973 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.765021 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":632}
I20260926 04:03:19.765431 7474 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 02076a495c44497996d0923adfd9de22 (127.0.200.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.765581 7523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 02076a495c44497996d0923adfd9de22 (127.0.200.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.766362 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.766431 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:19.766484 7474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.766521 7474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:19.766652 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.765325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45946)
0926 04:03:19.766596 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.766626 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1257}
W20260926 04:03:19.766950 6902 consensus_peers.cc:487] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee -> Peer 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188:45769): Couldn't send request to peer 1aef1e65b86443d795d601e47f86fbd0. Status: Network error: Client connection negotiation failed: client connection to 127.0.200.188:45769: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 04:03:19.767062 7663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.767210 7663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:03:19.775776 7567 master.cc:552] Master@127.0.200.188:45769 shutdown complete.
I20260926 04:03:19.775835 7567 master_runner.cc:305] Clearing existing system tablet
I20260926 04:03:19.775964 7567 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.776002 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.776750 7567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:19.777261 7666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.777498 7567 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.777555 7567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:03:19.777578 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "1aef1e65b86443d795d601e47f86fbd0"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "c4bfefe1c7c8969c9f77049be2d6fc36"
server_key_iv: "ad2cde0d7d888af19a16220fc9fb35e7"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.778335 7567 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:19.779860 7567 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 04:03:19.779929 7567 log.cc:1196] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 04:03:19.780117 7567 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Deleting consensus metadata
I20260926 04:03:19.780261 7567 master_runner.cc:315] Copying system tablet from 127.0.200.190:33111
I20260926 04:03:19.780862 7567 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: tablet copy: Beginning tablet copy session from remote peer at address 127.0.200.190:33111
I20260926 04:03:19.788384 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.781235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.0.200.190:33111)
0926 04:03:19.781281 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.781283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.781422 (+ 139us) server_negotiation.cc:407] Connection header received
0926 04:03:19.781488 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.781489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.781500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.781556 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.781936 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.782716 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.783746 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.784071 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.784157 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.785377 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.785378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.785379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.785386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.787126 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.787539 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.787541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.787541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.787603 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.787822 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.787822 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.787823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.787957 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.788023 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.788211 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.788288 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:19.788395 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.781166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36044)
0926 04:03:19.781306 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.781317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.781398 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.781582 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.781583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.781788 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.781849 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.781851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.782804 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.782806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.783645 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.783648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.784168 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.784169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.784232 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.784793 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.784798 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.785279 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.787214 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.787215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.787216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.787458 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.787678 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.787679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.787679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.787754 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.788030 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.788031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.788081 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.788166 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.788282 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260926 04:03:19.789194 6935 tablet_copy_service.cc:140] P 45c0700ef6f440488586dac0ef1f14ee: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1aef1e65b86443d795d601e47f86fbd0 ({username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:36044)
I20260926 04:03:19.789260 6935 tablet_copy_service.cc:161] P 45c0700ef6f440488586dac0ef1f14ee: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1aef1e65b86443d795d601e47f86fbd0 at {username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:36044: session id = 1aef1e65b86443d795d601e47f86fbd0-00000000000000000000000000000000
I20260926 04:03:19.789865 6935 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 04:03:19.790333 7567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 04:03:19.791680 7567 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: tablet copy: Starting download of 0 data blocks...
I20260926 04:03:19.791834 7567 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: tablet copy: Starting download of 1 WAL segments...
I20260926 04:03:19.792501 7567 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 04:03:19.793519 6935 tablet_copy_service.cc:342] P 45c0700ef6f440488586dac0ef1f14ee: Request end of tablet copy session 1aef1e65b86443d795d601e47f86fbd0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:36044
I20260926 04:03:19.793591 6935 tablet_copy_service.cc:433] P 45c0700ef6f440488586dac0ef1f14ee: ending tablet copy session 1aef1e65b86443d795d601e47f86fbd0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1aef1e65b86443d795d601e47f86fbd0
I20260926 04:03:19.796291 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.797683 7673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.797834 7674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.797660 7676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.799665 7567 server_base.cc:1034] running on GCE node
I20260926 04:03:19.799778 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.799940 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.801060 7567 hybrid_clock.cc:648] HybridClock initialized: now 1790395399801051 us; error 24 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.802666 7567 init.cc:377] Logged in from keytab as oryx/127.0.200.188@KRBTEST.COM (short username oryx)
I20260926 04:03:19.802951 7567 webserver.cc:460] Webserver started at http://127.0.200.188:34699/ using document root <none> and password file <none>
I20260926 04:03:19.803077 7567 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.803126 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.803978 7567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:19.804636 7683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.804821 7567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:19.804881 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "1aef1e65b86443d795d601e47f86fbd0"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "c4bfefe1c7c8969c9f77049be2d6fc36"
server_key_iv: "ad2cde0d7d888af19a16220fc9fb35e7"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.804945 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.822201 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.824112 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.827301 7567 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.188:45769
I20260926 04:03:19.827657 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
I20260926 04:03:19.828629 7736 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:03:19.829226 7736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Bootstrap starting.
I20260926 04:03:19.830682 7735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.188:45769 every 8 connection(s)
I20260926 04:03:19.833031 7736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:03:19.833436 7736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Bootstrap complete.
I20260926 04:03:19.833778 7736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } }
I20260926 04:03:19.833969 7736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1aef1e65b86443d795d601e47f86fbd0, State: Initialized, Role: LEARNER
I20260926 04:03:19.834053 7736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } }
I20260926 04:03:19.834336 7736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:19.834362 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:19.834467 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: LEARNER
I20260926 04:03:19.836410 7750 catalog_manager.cc:1259] Loaded cluster ID: c3823ba6a43f404abb464f2ff91f1170
I20260926 04:03:19.836462 7750 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: loading cluster ID for follower catalog manager: success
I20260926 04:03:19.837525 7750 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: acquiring CA information for follower catalog manager: success
I20260926 04:03:19.837841 7750 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 04:03:19.852756 7661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.853018 7661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.853060 7661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.854413 7661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.854483 7661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.854529 7661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:19.856320 7661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:45461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=35505
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:19.856619 7661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:19.856858 7661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:19.862031 7758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.862046 7755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:19.862049 7756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:19.862198 7661 server_base.cc:1034] running on GCE node
I20260926 04:03:19.862375 7661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:19.862668 7661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:19.864159 7661 hybrid_clock.cc:648] HybridClock initialized: now 1790395399864141 us; error 28 us; skew 500 ppm
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:19.867261 7661 init.cc:377] Logged in from keytab as oryx/127.0.200.129@KRBTEST.COM (short username oryx)
I20260926 04:03:19.867770 7661 webserver.cc:460] Webserver started at http://127.0.200.129:35505/ using document root <none> and password file <none>
I20260926 04:03:19.868033 7661 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:19.868090 7661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:19.869462 7661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:19.870100 7765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.870280 7661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:19.870384 7661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b2b520e255af42ec9730fa5b161178b8"
format_stamp: "Formatted at 2026-09-26 04:03:18 on dist-test-slave-j8c1"
server_key: "5f7d76dd9f66c3c0632c2e15fcdf23ff"
server_key_iv: "c76d421e2c7b9fd6fe7488993960c85e"
server_key_version: "encryptionkey@0"
I20260926 04:03:19.870793 7661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:19.887146 7661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:19.889590 7661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:19.889765 7661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:19.890022 7661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:19.890337 7661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:19.890393 7661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.890437 7661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:19.890489 7661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:19.897379 7661 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:45461
I20260926 04:03:19.897466 7878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:45461 every 8 connection(s)
I20260926 04:03:19.897866 7661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:19.904098 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7661
I20260926 04:03:19.904275 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7111
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
Sep 26 04:03:19 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395399, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:19.909992 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.899139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:43417 (local address 127.0.200.189:43611)
0926 04:03:19.899190 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.899192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.899853 (+ 661us) server_negotiation.cc:407] Connection header received
0926 04:03:19.899944 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.899946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.899966 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.900021 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.900788 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901305 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.902371 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.902637 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.902710 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.906404 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.906407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.906407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.906416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.908436 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.908897 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.908899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.908899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.908966 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.909219 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.909220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.909220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.909391 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.909460 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.909744 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.909844 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 04:03:19.910027 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.899062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.200.129:43417)
0926 04:03:19.899792 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.899802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.899878 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.900062 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.900063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.900475 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.900689 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.900694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901422 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.901428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.902270 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.902272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.902719 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.902719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.902859 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.903483 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.903494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.906246 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.908542 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.908549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.908552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.908810 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.909043 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.909044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.909045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.909154 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.909464 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.909468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.909602 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.909696 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.909837 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":86,"threads_started":1}
I20260926 04:03:19.910609 7881 heartbeater.cc:347] Connected to a master server at 127.0.200.189:43611
I20260926 04:03:19.910715 7881 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:19.910939 7881 heartbeater.cc:511] Master 127.0.200.189:43611 requested a full tablet report, sending...
I20260926 04:03:19.911304 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:39859
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:33111
--webserver_port=35611
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:03:19.911506 7513 ts_manager.cc:194] Registered new tserver with Master: b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461)
I20260926 04:03:19.914335 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.898721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:52865 (local address 127.0.200.188:45769)
0926 04:03:19.898844 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.898848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.899335 (+ 487us) server_negotiation.cc:407] Connection header received
0926 04:03:19.899902 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.899904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.899921 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.899998 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.900812 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901840 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.902815 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.903156 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.903225 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.911322 (+ 8097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.911324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.911324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.911333 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.912974 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913368 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.913370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.913370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.913439 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913666 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.913667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.913667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.913811 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.913854 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.914141 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.914244 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260926 04:03:19.914718 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.898754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.200.129:52865)
0926 04:03:19.899238 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.899248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.899667 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.900001 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.900002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.900541 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.900690 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.900696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901928 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.901929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.902740 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.902741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.903236 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.903236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.903297 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.903724 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.903735 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.911212 (+ 7477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.913083 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.913084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.913286 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.913513 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.913514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.913576 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.913912 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.913913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.913988 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.914429 (+ 441us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.914591 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":58,"thread_start_us":77,"threads_started":1}
I20260926 04:03:19.914881 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.899084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:59257 (local address 127.0.200.190:33111)
0926 04:03:19.899134 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:19.899136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:19.899147 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:19.899710 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:19.899711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:19.899725 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:19.899787 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:19.900806 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901569 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.902626 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.902950 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.903015 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.910951 (+ 7936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:19.910953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:19.910954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:19.910964 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:19.913521 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913909 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.913910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.913911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.913975 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:19.914204 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:19.914205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:19.914205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:19.914383 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:19.914458 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:19.914672 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 04:03:19.914760 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:03:19.915186 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:19.898754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:59257)
0926 04:03:19.898934 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:19.898971 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:19.899600 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:19.899935 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:19.899939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:19.900547 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:19.900690 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.900696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.901664 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.901665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.902524 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:19.902526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:19.903033 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:19.903034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:19.903123 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:19.903557 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:19.903562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:19.910858 (+ 7296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:19.913606 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.913607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.913607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.913801 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.914060 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:19.914061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:19.914061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:19.914132 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:19.914469 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:19.914470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:19.914512 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 04:03:19.914962 (+ 450us) client_negotiation.cc:240] Negotiation successful
0926 04:03:19.915063 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260926 04:03:19.915452 7880 heartbeater.cc:347] Connected to a master server at 127.0.200.188:45769
I20260926 04:03:19.915509 7880 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:19.915582 7879 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:19.915594 7880 heartbeater.cc:511] Master 127.0.200.188:45769 requested a full tablet report, sending...
I20260926 04:03:19.915745 7879 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:19.915865 7879 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:19.916136 7700 ts_manager.cc:194] Registered new tserver with Master: b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461)
I20260926 04:03:19.916193 6914 ts_manager.cc:194] Re-registered known tserver with Master: b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461)
I20260926 04:03:19.916604 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:59257
W20260926 04:03:19.997691 7887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:19.997951 7887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:19.997994 7887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:19.999358 7887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:19.999437 7887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:19.999486 7887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:20.001042 7887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:39859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=35611
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:20.001329 7887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:20.001582 7887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:20.007367 7893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:20.007431 7895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:20.007365 7892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:20.007545 7887 server_base.cc:1034] running on GCE node
I20260926 04:03:20.007870 7887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:20.008239 7887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:20.009405 7887 hybrid_clock.cc:648] HybridClock initialized: now 1790395400009392 us; error 26 us; skew 500 ppm
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:20.012072 7887 init.cc:377] Logged in from keytab as oryx/127.0.200.130@KRBTEST.COM (short username oryx)
I20260926 04:03:20.012620 7887 webserver.cc:460] Webserver started at http://127.0.200.130:35611/ using document root <none> and password file <none>
I20260926 04:03:20.012862 7887 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:20.012948 7887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:20.014400 7887 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:03:20.015239 7902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.015497 7887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:20.015602 7887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "ce99795d5e014de183abc4b49576e3d2"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "04d6c71abb4618bec6431b245e5dbc41"
server_key_iv: "9d34b87f0aa81a4456844f0e2f456144"
server_key_version: "encryptionkey@0"
I20260926 04:03:20.016009 7887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:20.029462 7887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:20.031831 7887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:20.032088 7887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:20.032344 7887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:20.032665 7887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:20.032722 7887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.032765 7887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:20.032793 7887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.039152 7887 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:39859
I20260926 04:03:20.039187 8015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:39859 every 8 connection(s)
I20260926 04:03:20.039569 7887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:20.047870 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 7887
I20260926 04:03:20.048048 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7245
I20260926 04:03:20.052373 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:56053 (local address 127.0.200.190:33111)
0926 04:03:20.040340 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.040342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.040594 (+ 252us) server_negotiation.cc:407] Connection header received
0926 04:03:20.041227 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.041228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.041242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.041298 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.042132 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.043011 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044291 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.044597 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044667 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.048771 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.048773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.048773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.048781 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.050446 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051050 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.051052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.051053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.051127 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051451 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.051452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.051453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.051617 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.051670 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.052139 (+ 469us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.052240 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:03:20.052590 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:51699 (local address 127.0.200.189:43611)
0926 04:03:20.040816 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.040818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.041269 (+ 451us) server_negotiation.cc:407] Connection header received
0926 04:03:20.041398 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.041399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.041411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.041460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.042115 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.042739 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.043762 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.043973 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044060 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.047069 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.047071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.047071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.047078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.051300 (+ 4222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051720 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.051721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.051721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.051783 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051995 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.051997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.051998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.052145 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.052191 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.052403 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.052502 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:03:20.052975 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.200.130:51699)
0926 04:03:20.041212 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.041222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.041294 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.041473 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.041474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.041811 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.042010 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.042014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.042841 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.042844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.043669 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.043671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.044071 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.044223 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.044889 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.044897 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.046950 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.051388 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.051390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.051618 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.051857 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.051858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.051916 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.052246 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.052247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.052288 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.052704 (+ 416us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.052795 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":52,"threads_started":1}
I20260926 04:03:20.052984 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:56053)
0926 04:03:20.040498 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.040510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.041073 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.041386 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.041391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.041911 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.042011 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.042017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.043121 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.043122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.044182 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.044184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.044733 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.044809 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.045284 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.045290 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.048687 (+ 3397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.050617 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.050626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.050630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.050938 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.051202 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.051203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.051204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.051336 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.051724 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.051731 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.051928 (+ 197us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.052488 (+ 560us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.052696 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":28,"spinlock_wait_cycles":8960,"thread_start_us":108,"threads_started":1}
I20260926 04:03:20.053426 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:59731 (local address 127.0.200.188:45769)
0926 04:03:20.040340 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.040342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.040582 (+ 240us) server_negotiation.cc:407] Connection header received
0926 04:03:20.041160 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.041162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.041180 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.041287 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.042132 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.043055 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044251 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.044568 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044647 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.047550 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.047552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.047552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.047560 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.052141 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.052567 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.052569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.052570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.052645 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.052832 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.052833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.052834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.052959 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.052999 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.053185 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.053282 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:03:20.053458 8016 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:20.053583 8016 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.053789 8016 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:20.053854 8018 heartbeater.cc:347] Connected to a master server at 127.0.200.189:43611
I20260926 04:03:20.053887 8018 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.053938 8018 heartbeater.cc:511] Master 127.0.200.189:43611 requested a full tablet report, sending...
I20260926 04:03:20.054020 8020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.040303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.200.130:59731)
0926 04:03:20.040461 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.040504 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.041041 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.041386 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.041391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.041826 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.042012 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.042017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.043149 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.043150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.044150 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.044152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.044657 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.044658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.044731 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.045099 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.045103 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.047450 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.052235 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.052236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.052237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.052472 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.052712 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.052713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.052713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.052763 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.053042 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.053043 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.053082 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.053508 (+ 426us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.053639 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":53,"thread_start_us":86,"threads_started":1}
I20260926 04:03:20.054334 8017 heartbeater.cc:347] Connected to a master server at 127.0.200.188:45769
I20260926 04:03:20.054340 6914 ts_manager.cc:194] Re-registered known tserver with Master: ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:20.054391 8017 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.054452 8017 heartbeater.cc:511] Master 127.0.200.188:45769 requested a full tablet report, sending...
I20260926 04:03:20.054608 7513 ts_manager.cc:194] Registered new tserver with Master: ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:20.054773 7700 ts_manager.cc:194] Registered new tserver with Master: ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:20.054908 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:56053
I20260926 04:03:20.055688 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:40151
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:33111
--webserver_port=45505
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:20.139536 8023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:20.139758 8023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:20.139786 8023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:20.141055 8023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:20.141112 8023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:20.141142 8023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:20.142717 8023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:40151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=45505
--tserver_master_addrs=127.0.200.190:33111,127.0.200.189:43611,127.0.200.188:45769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:20.142938 8023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:20.143139 8023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:20.148633 8028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:20.148653 8031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:20.148635 8029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:20.148914 8023 server_base.cc:1034] running on GCE node
I20260926 04:03:20.149255 8023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:20.149535 8023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:20.150713 8023 hybrid_clock.cc:648] HybridClock initialized: now 1790395400150705 us; error 40 us; skew 500 ppm
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:20.153635 8023 init.cc:377] Logged in from keytab as oryx/127.0.200.131@KRBTEST.COM (short username oryx)
I20260926 04:03:20.154269 8023 webserver.cc:460] Webserver started at http://127.0.200.131:45505/ using document root <none> and password file <none>
I20260926 04:03:20.154542 8023 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:20.154701 8023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:20.156222 8023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:20.156971 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.157281 8023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:20.157390 8023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "72f2ba1281ba43499bf09b79bfa073f1"
format_stamp: "Formatted at 2026-09-26 04:03:19 on dist-test-slave-j8c1"
server_key: "f1f439cae4c43a8e54b8978580b05174"
server_key_iv: "3265bec17820503fb593585a69085e68"
server_key_version: "encryptionkey@0"
I20260926 04:03:20.157792 8023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:20.176668 8023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:20.179332 8023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:20.179513 8023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:20.179776 8023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:20.180100 8023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:20.180157 8023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.180203 8023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:20.180231 8023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:20.185999 8023 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:40151
I20260926 04:03:20.186090 8151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:40151 every 8 connection(s)
I20260926 04:03:20.186471 8023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:20.191217 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8023
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.189@KRBTEST.COM
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.188@KRBTEST.COM
Sep 26 04:03:20 dist-test-slave-j8c1 krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395400, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:20.199743 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:44591 (local address 127.0.200.188:45769)
0926 04:03:20.187485 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.187488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.187741 (+ 253us) server_negotiation.cc:407] Connection header received
0926 04:03:20.188410 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.188411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.188424 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.188488 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.190314 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.190936 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.191901 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.192246 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.192293 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.195848 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.195849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.195850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.195857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.197673 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.198240 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.198242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.198242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.198310 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.198654 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.198655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.198655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.198802 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.198867 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.199532 (+ 665us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.199629 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260926 04:03:20.199862 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:48169 (local address 127.0.200.190:33111)
0926 04:03:20.187402 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.187404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.187796 (+ 392us) server_negotiation.cc:407] Connection header received
0926 04:03:20.188438 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.188439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.188453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.188485 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.189330 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.190195 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.192697 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.193017 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.193064 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.196648 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.196650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.196650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.196660 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.198420 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.198867 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.198868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.198868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.198932 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.199164 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.199166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.199166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.199320 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.199381 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.199666 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.199750 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:03:20.200219 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.200.131:44591)
0926 04:03:20.187563 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.187622 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.188285 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.188572 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.188581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.188980 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.189221 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.189226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.191029 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.191030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.191780 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.191782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.192376 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.192377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.192502 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.193373 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.193378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.195747 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.197782 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.197790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.197793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.198142 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.198387 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.198388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.198388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.198539 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.198895 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.198903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.199120 (+ 217us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.199244 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.199916 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260926 04:03:20.200711 8153 heartbeater.cc:347] Connected to a master server at 127.0.200.188:45769
I20260926 04:03:20.200805 8153 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.200995 8153 heartbeater.cc:511] Master 127.0.200.188:45769 requested a full tablet report, sending...
I20260926 04:03:20.201249 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:59703 (local address 127.0.200.189:43611)
0926 04:03:20.188001 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.188003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.188011 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:20.188438 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.188440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.188453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.188485 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.189370 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.190241 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.191917 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.192218 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.192263 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.196624 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.196625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.196625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.196633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.199135 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.199901 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.199902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.199903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.199968 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.200430 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.200431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.200431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.200571 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.200632 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.201059 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.201148 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 04:03:20.201454 7700 ts_manager.cc:194] Registered new tserver with Master: 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131:40151)
I20260926 04:03:20.201009 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.200.131:59703)
0926 04:03:20.187896 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.187906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.188320 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.188588 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.188589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.189009 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.189223 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.189228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.190344 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.190345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.191769 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.191772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.192357 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.192358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.192502 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.193266 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.193277 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.196534 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.199232 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.199233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.199234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.199817 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.200037 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.200038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.200038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.200351 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.200660 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.200661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.200708 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.200771 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.200872 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":98,"thread_start_us":76,"threads_started":1}
I20260926 04:03:20.202250 8154 heartbeater.cc:347] Connected to a master server at 127.0.200.189:43611
I20260926 04:03:20.202298 8154 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.202359 8154 heartbeater.cc:511] Master 127.0.200.189:43611 requested a full tablet report, sending...
I20260926 04:03:20.202376 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.192740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.0.200.189:43611)
0926 04:03:20.192864 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.192866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.194535 (+ 1669us) server_negotiation.cc:407] Connection header received
0926 04:03:20.194693 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.194694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.194708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.194751 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.195485 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.196270 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.197363 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.197710 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.197754 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.199305 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.199306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.199307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.199313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.201136 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.201527 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.201528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.201528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.201585 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.201798 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.201799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.201799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.201924 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.202065 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.202173 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.202253 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":82,"threads_started":1}
I20260926 04:03:20.202654 7513 ts_manager.cc:194] Registered new tserver with Master: 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131:40151)
I20260926 04:03:20.202802 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.187330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:48169)
0926 04:03:20.187731 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.187744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.188320 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.188944 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.188945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.189098 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:20.189222 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.189228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.190344 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.190345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.192584 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.192586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.193150 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.193151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.193239 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.193702 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:20.193707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:20.196539 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:20.198515 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.198516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.198517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.198763 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.199005 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:20.199006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:20.199006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:20.199080 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:20.199411 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:20.199412 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:20.199458 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.199559 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.202675 (+ 3116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":3243,"thread_start_us":89,"threads_started":1}
I20260926 04:03:20.203341 8152 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33111
I20260926 04:03:20.203397 8152 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:20.203483 8152 heartbeater.cc:511] Master 127.0.200.190:33111 requested a full tablet report, sending...
I20260926 04:03:20.203706 6914 ts_manager.cc:194] Re-registered known tserver with Master: 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131:40151)
I20260926 04:03:20.204133 6914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:48169
I20260926 04:03:20.209941 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.193579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.0.200.188:45769)
0926 04:03:20.194761 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.194764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.194772 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:20.194794 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.194794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.194806 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.194868 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.195625 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.203600 (+ 7975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.204716 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.205014 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.205102 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.206497 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.206499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.206499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.206507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.208554 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.209021 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.209022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.209022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.209105 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.209338 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.209339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.209339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.209488 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.209579 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.209747 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.209838 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1160,"thread_start_us":64,"threads_started":1}
I20260926 04:03:20.209995 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.192258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36054 (local address 127.0.200.190:33111)
0926 04:03:20.194305 (+ 2047us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.194307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.194436 (+ 129us) server_negotiation.cc:407] Connection header received
0926 04:03:20.194576 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.194577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.194591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.194641 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.196899 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.202616 (+ 5717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.203823 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.204185 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.204230 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.206353 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.206355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.206355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.206361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.208466 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.209021 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.209022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.209023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.209106 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.209338 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.209338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.209339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.209483 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.209580 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.209747 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.209838 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2003,"thread_start_us":81,"threads_started":1}
I20260926 04:03:20.211684 6914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:20.213244 6914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:20.221119 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.217315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60946 (local address 127.0.200.131:40151)
0926 04:03:20.217473 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.217477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.217496 (+ 19us) server_negotiation.cc:407] Connection header received
0926 04:03:20.217533 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.217535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.217549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.217642 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.217761 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.218746 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220265 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.220762 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220857 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.220918 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.220995 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.221030 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260926 04:03:20.221108 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.216904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:40151 (local address 127.0.0.1:60946)
0926 04:03:20.217077 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.217089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.217316 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.217645 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.217646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.217648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.217700 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.217701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.218841 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.218841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.220177 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.220179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.220859 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.220918 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.220923 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.220981 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.221030 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2}
I20260926 04:03:20.221252 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.216994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:39859 (local address 127.0.0.1:55604)
0926 04:03:20.217189 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.217197 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.217228 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.218013 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.218014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.218016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.218046 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.218047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.219215 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.219217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.220066 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.220068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.220971 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.221037 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.221040 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.221070 (+ 30us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.221111 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260926 04:03:20.221585 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.217165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.0.200.129:45461)
0926 04:03:20.217384 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.217388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.217397 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:20.217439 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.217441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.217459 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.217545 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.217724 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.218891 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220138 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.220872 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220944 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.221014 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.221457 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.221503 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
I20260926 04:03:20.221583 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.217491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.0.200.130:39859)
0926 04:03:20.217849 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.217854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.217862 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:20.217897 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.217899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.217914 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.217992 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.218104 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.219156 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.220178 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.220872 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.221159 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.221446 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.221470 (+ 24us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.221508 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":93,"threads_started":1}
I20260926 04:03:20.221964 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.216944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45461 (local address 127.0.0.1:37548)
0926 04:03:20.217128 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.217140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.217192 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.217559 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.217560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.217563 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.217623 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.217626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.218967 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.218968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.220042 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.220045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.221044 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.221044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.221096 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.221099 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.221157 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.221189 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260926 04:03:20.222020 8086 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf (DEFAULT_TABLE table=test-table [id=285aac55b3224d99869d19624db1678d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:20.222417 8086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9256a4b4164d44a1b75ccd356ac0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:20.222630 7813 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf (DEFAULT_TABLE table=test-table [id=285aac55b3224d99869d19624db1678d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:20.222954 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9256a4b4164d44a1b75ccd356ac0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:20.222973 7950 tablet_service.cc:1467] Processing CreateTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf (DEFAULT_TABLE table=test-table [id=285aac55b3224d99869d19624db1678d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:20.223280 7950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b9256a4b4164d44a1b75ccd356ac0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:20.226501 8178 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Bootstrap starting.
I20260926 04:03:20.226540 8179 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Bootstrap starting.
I20260926 04:03:20.227187 8179 tablet_bootstrap.cc:654] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:20.227190 8178 tablet_bootstrap.cc:654] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:20.227478 8178 log.cc:824] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:20.227478 8179 log.cc:824] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:20.227823 8180 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Bootstrap starting.
I20260926 04:03:20.228302 8179 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: No bootstrap required, opened a new log
I20260926 04:03:20.228307 8178 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: No bootstrap required, opened a new log
I20260926 04:03:20.228374 8178 ts_tablet_manager.cc:1397] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:20.228403 8179 ts_tablet_manager.cc:1397] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:20.229714 8178 raft_consensus.cc:348] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.229853 8178 raft_consensus.cc:374] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:20.229876 8178 raft_consensus.cc:729] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f2ba1281ba43499bf09b79bfa073f1, State: Initialized, Role: FOLLOWER
I20260926 04:03:20.230006 8178 consensus_queue.cc:260] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.230291 8178 ts_tablet_manager.cc:1428] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:20.230501 8152 heartbeater.cc:503] Master 127.0.200.190:33111 was elected leader, sending a full tablet report...
I20260926 04:03:20.230427 8179 raft_consensus.cc:348] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.230638 8179 raft_consensus.cc:374] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:20.230681 8179 raft_consensus.cc:729] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b520e255af42ec9730fa5b161178b8, State: Initialized, Role: FOLLOWER
I20260926 04:03:20.230801 8179 consensus_queue.cc:260] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.231069 8179 ts_tablet_manager.cc:1428] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:03:20.231104 7879 heartbeater.cc:503] Master 127.0.200.190:33111 was elected leader, sending a full tablet report...
I20260926 04:03:20.232498 8180 tablet_bootstrap.cc:654] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:20.232821 8180 log.cc:824] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:20.233599 8180 tablet_bootstrap.cc:492] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: No bootstrap required, opened a new log
I20260926 04:03:20.233659 8180 ts_tablet_manager.cc:1397] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 04:03:20.235836 8180 raft_consensus.cc:348] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.235960 8180 raft_consensus.cc:374] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:20.235980 8180 raft_consensus.cc:729] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce99795d5e014de183abc4b49576e3d2, State: Initialized, Role: FOLLOWER
I20260926 04:03:20.236068 8180 consensus_queue.cc:260] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.236279 8180 ts_tablet_manager.cc:1428] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:03:20.236317 8016 heartbeater.cc:503] Master 127.0.200.190:33111 was elected leader, sending a full tablet report...
W20260926 04:03:20.290373 8019 tablet.cc:2369] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:20.329969 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.327412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.0.200.188:45769)
0926 04:03:20.327478 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.327480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.327488 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:20.327538 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.327539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.327558 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.327633 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.327787 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.328297 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.329212 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.329632 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.329719 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.329805 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.329849 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.329889 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:03:20.329969 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.327271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45968)
0926 04:03:20.327334 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.327347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.327397 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.327641 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.327642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.327644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.327698 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.327700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.328387 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.328388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.329114 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.329116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.329721 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.329722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.329782 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.329785 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.329842 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.329891 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 04:03:20.330492 7710 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260926 04:03:20.331457 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c0700ef6f440488586dac0ef1f14ee. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:20.331571 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: LEARNER
W20260926 04:03:20.399243 7882 tablet.cc:2369] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:03:20.437511 8155 tablet.cc:2369] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:20.456782 8183 raft_consensus.cc:482] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:20.456918 8183 raft_consensus.cc:504] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.457378 8183 leader_election.cc:288] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461), ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:20.460412 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.457724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:45073 (local address 127.0.200.129:45461)
0926 04:03:20.457808 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.457811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.457821 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:20.457875 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.457876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.457884 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.457961 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.458114 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.458669 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.459674 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.460047 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.460136 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.460146 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.460290 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.460328 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 04:03:20.460441 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.457743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:44339 (local address 127.0.200.130:39859)
0926 04:03:20.457794 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.457796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.457805 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:20.457835 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.457836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.457845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.457925 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:20.458115 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.458625 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.459686 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.460083 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.460167 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.460175 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.460333 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.460369 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:20.460443 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.457609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45461 (local address 127.0.200.131:45073)
0926 04:03:20.457752 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.457764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.457809 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.457963 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.457964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.457965 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.458039 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.458040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.458769 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.458770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.459573 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.459576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.460135 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.460136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.460212 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.460215 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.460284 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.460328 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":104}
I20260926 04:03:20.460456 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.457609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:39859 (local address 127.0.200.131:44339)
0926 04:03:20.457675 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:20.457689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:20.457763 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:20.457928 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:20.457929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:20.457931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:20.458015 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.458018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.458714 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.458715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.459579 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:20.459581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.460166 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.460167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:20.460247 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.460250 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:20.460325 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 04:03:20.460364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":23}
I20260926 04:03:20.460811 7833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9256a4b4164d44a1b75ccd356ac0cf" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b520e255af42ec9730fa5b161178b8" is_pre_election: true
I20260926 04:03:20.460986 7833 raft_consensus.cc:2454] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 0.
I20260926 04:03:20.461196 7970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9256a4b4164d44a1b75ccd356ac0cf" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce99795d5e014de183abc4b49576e3d2" is_pre_election: true
I20260926 04:03:20.461282 8039 leader_election.cc:302] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f2ba1281ba43499bf09b79bfa073f1, b2b520e255af42ec9730fa5b161178b8; no voters:
I20260926 04:03:20.461367 7970 raft_consensus.cc:2454] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 0.
I20260926 04:03:20.461436 8183 raft_consensus.cc:2781] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:20.461510 8183 raft_consensus.cc:482] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:20.461551 8183 raft_consensus.cc:3037] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:20.462406 8183 raft_consensus.cc:504] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.462602 8183 leader_election.cc:288] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 election: Requested vote from peers b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461), ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:20.462760 7970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9256a4b4164d44a1b75ccd356ac0cf" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce99795d5e014de183abc4b49576e3d2"
I20260926 04:03:20.462838 7970 raft_consensus.cc:3037] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:20.463949 7970 raft_consensus.cc:2454] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 1.
I20260926 04:03:20.464092 7833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b9256a4b4164d44a1b75ccd356ac0cf" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b520e255af42ec9730fa5b161178b8"
I20260926 04:03:20.464185 7833 raft_consensus.cc:3037] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:20.464228 8041 leader_election.cc:302] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f2ba1281ba43499bf09b79bfa073f1, ce99795d5e014de183abc4b49576e3d2; no voters:
I20260926 04:03:20.464308 8183 raft_consensus.cc:2781] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:20.464459 8183 raft_consensus.cc:686] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 LEADER]: Becoming Leader. State: Replica: 72f2ba1281ba43499bf09b79bfa073f1, State: Running, Role: LEADER
I20260926 04:03:20.464560 8183 consensus_queue.cc:237] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:20.465168 7833 raft_consensus.cc:2454] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 1.
I20260926 04:03:20.465787 6914 catalog_manager.cc:5495] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "72f2ba1281ba43499bf09b79bfa073f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:20.466379 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: NON_VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: true } } }
I20260926 04:03:20.466465 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:20.467257 7475 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: attempting to promote NON_VOTER 1aef1e65b86443d795d601e47f86fbd0 to VOTER
I20260926 04:03:20.467393 7475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:20.467777 7710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 LEARNER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 04:03:20.467928 7523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Refusing update from remote peer 45c0700ef6f440488586dac0ef1f14ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260926 04:03:20.467972 7664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 04:03:20.468137 7664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260926 04:03:20.468696 7475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.468988 7523 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.469558 7710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.469683 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.469758 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:20.469815 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.469887 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:20.469941 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.469995 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:20.470079 8195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.470224 8195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:03:20.470753 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45c0700ef6f440488586dac0ef1f14ee" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:20.470871 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:03:20.504177 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.501485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.0.200.131:40151)
0926 04:03:20.501559 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.501562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.501570 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:20.501616 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.501617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.501633 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.501712 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:20.501873 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.502449 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.503286 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.503488 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.503574 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.503778 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:20.503919 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:20.504031 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.504064 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.504088 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 04:03:20.506402 7970 raft_consensus.cc:1261] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Refusing update from remote peer 72f2ba1281ba43499bf09b79bfa073f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:20.506510 7833 raft_consensus.cc:1261] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Refusing update from remote peer 72f2ba1281ba43499bf09b79bfa073f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:20.506872 8191 consensus_queue.cc:1035] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:20.507047 8183 consensus_queue.cc:1035] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:20.509912 8197 mvcc.cc:205] Tried to move back new op lower bound from 7333459560470945792 to 7333459560303300608. Current Snapshot: MvccSnapshot[applied={T|T < 7333459560470945792}]
I20260926 04:03:20.510062 8199 mvcc.cc:205] Tried to move back new op lower bound from 7333459560470945792 to 7333459560303300608. Current Snapshot: MvccSnapshot[applied={T|T < 7333459560470945792}]
I20260926 04:03:20.511536 8198 mvcc.cc:205] Tried to move back new op lower bound from 7333459560470945792 to 7333459560303300608. Current Snapshot: MvccSnapshot[applied={T|T < 7333459560470945792}]
I20260926 04:03:20.512584 6914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:20.512686 6914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:20.513991 6914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45c0700ef6f440488586dac0ef1f14ee: Sending DeleteTablet for 3 replicas of tablet 9b9256a4b4164d44a1b75ccd356ac0cf
I20260926 04:03:20.514338 8086 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:20 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:60946
I20260926 04:03:20.514367 7950 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:20 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55604
I20260926 04:03:20.514422 7813 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b9256a4b4164d44a1b75ccd356ac0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:20 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:37548
I20260926 04:03:20.514645 8206 tablet_replica.cc:330] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: stopping tablet replica
I20260926 04:03:20.514742 8206 raft_consensus.cc:2229] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:20.514837 8206 raft_consensus.cc:2258] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:20.515309 8206 ts_tablet_manager.cc:1905] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:20.515395 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 6883
I20260926 04:03:20.515434 8208 tablet_replica.cc:330] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: stopping tablet replica
I20260926 04:03:20.516817 8208 raft_consensus.cc:2229] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:20.516952 8208 raft_consensus.cc:2258] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:20.517431 8208 ts_tablet_manager.cc:1905] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:20.517653 8207 tablet_replica.cc:330] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: stopping tablet replica
I20260926 04:03:20.517783 8207 raft_consensus.cc:2229] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:20.517894 8207 raft_consensus.cc:2258] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:20.518352 8207 ts_tablet_manager.cc:1905] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:20.518936 8206 ts_tablet_manager.cc:1918] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:20.519002 8206 log.cc:1196] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/9b9256a4b4164d44a1b75ccd356ac0cf
I20260926 04:03:20.519284 8206 ts_tablet_manager.cc:1939] T 9b9256a4b4164d44a1b75ccd356ac0cf P 72f2ba1281ba43499bf09b79bfa073f1: Deleting consensus metadata
I20260926 04:03:20.519304 8208 ts_tablet_manager.cc:1918] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:20.519367 8208 log.cc:1196] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/9b9256a4b4164d44a1b75ccd356ac0cf
I20260926 04:03:20.519615 8208 ts_tablet_manager.cc:1939] T 9b9256a4b4164d44a1b75ccd356ac0cf P b2b520e255af42ec9730fa5b161178b8: Deleting consensus metadata
I20260926 04:03:20.520881 8207 ts_tablet_manager.cc:1918] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:20.520957 8207 log.cc:1196] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/9b9256a4b4164d44a1b75ccd356ac0cf
I20260926 04:03:20.521224 8207 ts_tablet_manager.cc:1939] T 9b9256a4b4164d44a1b75ccd356ac0cf P ce99795d5e014de183abc4b49576e3d2: Deleting consensus metadata
W20260926 04:03:20.521759 7904 connection.cc:535] server connection from 127.0.0.1:55604 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 04:03:20.521776 7768 connection.cc:535] server connection from 127.0.0.1:37548 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 04:03:20.521960 8040 connection.cc:535] server connection from 127.0.0.1:60946 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 04:03:20.531280 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.515278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.0.200.188:45769)
0926 04:03:20.515320 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.515322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.522476 (+ 7154us) server_negotiation.cc:407] Connection header received
0926 04:03:20.522543 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.522566 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.522578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.522627 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.523506 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.524229 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.525356 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.525657 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.525703 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.527398 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.527400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.527400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.527408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.529773 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.530414 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.530416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.530416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.530469 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.530702 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.530703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.530703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.530826 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.530894 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.531081 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.531178 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:20.531700 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.523920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.0.200.189:43611)
0926 04:03:20.523967 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.523968 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.523975 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:03:20.523996 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.523996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.524007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.524051 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.524839 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.525622 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.526806 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.527143 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.527190 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.528651 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.528652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.528653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.528661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.530390 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.530811 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.530812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.530812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.530869 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.531075 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.531076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.531077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.531222 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.531303 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.531511 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.531602 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:03:20.532177 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:20.524336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.0.200.188:45769)
0926 04:03:20.524378 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:20.524379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:20.524487 (+ 108us) server_negotiation.cc:407] Connection header received
0926 04:03:20.524579 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:20.524579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:20.524588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:20.524618 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:20.525622 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.526348 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.527327 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:20.527583 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:20.527646 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:20.529127 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:20.529128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:20.529128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:20.529135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:20.531021 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.531371 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.531372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.531372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.531426 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:20.531621 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:20.531621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:20.531622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:20.531752 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:20.531825 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:20.532018 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 04:03:20.532089 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:03:21.511616 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:21.511352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:46261)
0926 04:03:21.511480 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:21.511561 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
W20260926 04:03:21.511979 8016 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:33111 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:21.512311 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:21.512081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:45565)
0926 04:03:21.512194 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:21.512257 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":52,"threads_started":1}
W20260926 04:03:21.512542 7879 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:33111 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:21.512892 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:21.512680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:53505)
0926 04:03:21.512811 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:21.512858 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":50,"threads_started":1}
W20260926 04:03:21.513062 8152 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:33111 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:22.014464 8219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45c0700ef6f440488586dac0ef1f14ee)
I20260926 04:03:22.014699 8220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 45c0700ef6f440488586dac0ef1f14ee)
I20260926 04:03:22.014683 8219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:22.014874 8220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:22.014961 8219 leader_election.cc:288] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 02076a495c44497996d0923adfd9de22 (127.0.200.189:43611)
I20260926 04:03:22.015245 8220 leader_election.cc:288] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188:45769)
I20260926 04:03:22.015619 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36536)
0926 04:03:22.015519 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.015569 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":64,"threads_started":1}
I20260926 04:03:22.015619 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36552)
0926 04:03:22.015507 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.015558 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":40,"threads_started":1}
W20260926 04:03:22.015736 7499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
W20260926 04:03:22.015736 7686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:22.016234 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.016055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36562)
0926 04:03:22.016167 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.016199 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260926 04:03:22.016306 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.016057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36564)
0926 04:03:22.016235 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.016272 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260926 04:03:22.016328 7686 leader_election.cc:334] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
W20260926 04:03:22.016383 7499 leader_election.cc:334] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:22.019096 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:45769 (local address 127.0.0.1:45994)
0926 04:03:22.015919 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.015931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:22.016088 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:22.016289 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:22.016290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:22.016292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:22.016454 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:22.016456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.017331 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.017332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:22.018144 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:22.018147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.018840 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:22.018905 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:22.018909 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:22.018963 (+ 54us) client_negotiation.cc:240] Negotiation successful
0926 04:03:22.019007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":109,"thread_start_us":56,"threads_started":1}
I20260926 04:03:22.019179 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.0.200.188:45769)
0926 04:03:22.015636 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:22.015638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:22.016144 (+ 506us) server_negotiation.cc:407] Connection header received
0926 04:03:22.016170 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:22.016171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:22.016182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:22.016244 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:22.016524 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.017226 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018263 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.018766 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018824 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:22.018834 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:22.018979 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 04:03:22.019026 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":42,"threads_started":1}
I20260926 04:03:22.019192 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:43611 (local address 127.0.0.1:50058)
0926 04:03:22.015601 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.015610 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:22.015670 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:22.016121 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:22.016123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:22.016125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:22.016193 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:22.016195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.017486 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.017487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:22.018325 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:22.018326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.018918 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:22.018993 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:22.018996 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:22.019052 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 04:03:22.019086 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":60,"threads_started":1}
I20260926 04:03:22.019232 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.015440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.0.200.189:43611)
0926 04:03:22.015962 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:22.015964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:22.015974 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:22.016001 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:22.016002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:22.016013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:22.016114 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:22.016277 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.017397 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018404 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:22.018842 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:22.018910 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:22.018920 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:22.019065 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 04:03:22.019098 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":36,"threads_started":1}
I20260926 04:03:22.019600 7523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1aef1e65b86443d795d601e47f86fbd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02076a495c44497996d0923adfd9de22" is_pre_election: true
I20260926 04:03:22.019609 7710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02076a495c44497996d0923adfd9de22" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1aef1e65b86443d795d601e47f86fbd0" is_pre_election: true
I20260926 04:03:22.019711 7710 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02076a495c44497996d0923adfd9de22 in term 1.
I20260926 04:03:22.019711 7523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aef1e65b86443d795d601e47f86fbd0 in term 1.
I20260926 04:03:22.019860 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02076a495c44497996d0923adfd9de22, 1aef1e65b86443d795d601e47f86fbd0; no voters: 45c0700ef6f440488586dac0ef1f14ee
I20260926 04:03:22.019879 7500 leader_election.cc:302] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02076a495c44497996d0923adfd9de22, 1aef1e65b86443d795d601e47f86fbd0; no voters: 45c0700ef6f440488586dac0ef1f14ee
I20260926 04:03:22.019973 8219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 04:03:22.019984 8220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 04:03:22.020015 8219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45c0700ef6f440488586dac0ef1f14ee)
I20260926 04:03:22.020021 8220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 45c0700ef6f440488586dac0ef1f14ee)
I20260926 04:03:22.020042 8219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:03:22.020044 8220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:03:22.020910 8220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:22.020910 8219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:22.021068 8219 leader_election.cc:288] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 election: Requested vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 02076a495c44497996d0923adfd9de22 (127.0.200.189:43611)
I20260926 04:03:22.021083 8220 leader_election.cc:288] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 election: Requested vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 1aef1e65b86443d795d601e47f86fbd0 (127.0.200.188:45769)
I20260926 04:03:22.021334 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.021215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36584)
0926 04:03:22.021284 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.021311 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 04:03:22.021518 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.021389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36588)
0926 04:03:22.021462 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.021491 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 04:03:22.021576 7523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1aef1e65b86443d795d601e47f86fbd0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02076a495c44497996d0923adfd9de22"
I20260926 04:03:22.021636 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.021559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36604)
0926 04:03:22.021593 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.021616 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 04:03:22.021565 7710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02076a495c44497996d0923adfd9de22" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1aef1e65b86443d795d601e47f86fbd0"
I20260926 04:03:22.021670 7523 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1aef1e65b86443d795d601e47f86fbd0 in current term 2: Already voted for candidate 02076a495c44497996d0923adfd9de22 in this term.
I20260926 04:03:22.021715 7710 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02076a495c44497996d0923adfd9de22 in current term 2: Already voted for candidate 1aef1e65b86443d795d601e47f86fbd0 in this term.
W20260926 04:03:22.021674 7499 leader_election.cc:334] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:22.021963 7500 leader_election.cc:302] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02076a495c44497996d0923adfd9de22; no voters: 1aef1e65b86443d795d601e47f86fbd0, 45c0700ef6f440488586dac0ef1f14ee
I20260926 04:03:22.022064 8220 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 04:03:22.022136 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.022023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36608)
0926 04:03:22.022078 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.022101 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 04:03:22.022197 7686 leader_election.cc:334] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:22.022234 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aef1e65b86443d795d601e47f86fbd0; no voters: 02076a495c44497996d0923adfd9de22, 45c0700ef6f440488586dac0ef1f14ee
I20260926 04:03:22.022310 8219 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 04:03:22.512616 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.512333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:48157)
0926 04:03:22.512514 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.512558 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":137,"threads_started":1}
I20260926 04:03:22.513082 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.512891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:37595)
0926 04:03:22.512995 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.513038 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":46,"threads_started":1}
I20260926 04:03:22.513563 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:22.513362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:58363)
0926 04:03:22.513490 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:22.513522 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260926 04:03:23.513463 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:23.513138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:37425)
0926 04:03:23.513308 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:23.513354 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260926 04:03:23.513688 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:23.513484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:34551)
0926 04:03:23.513610 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:23.513647 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260926 04:03:23.514112 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:23.513946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:35559)
0926 04:03:23.514033 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:23.514065 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":49,"threads_started":1}
I20260926 04:03:24.030313 8236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:24.030452 8236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:24.030697 8236 leader_election.cc:288] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 02076a495c44497996d0923adfd9de22 (127.0.200.189:43611)
I20260926 04:03:24.031071 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.030872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36862)
0926 04:03:24.030978 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.031020 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":68,"threads_started":1}
I20260926 04:03:24.031201 7523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1aef1e65b86443d795d601e47f86fbd0" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02076a495c44497996d0923adfd9de22" is_pre_election: true
I20260926 04:03:24.031315 7523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aef1e65b86443d795d601e47f86fbd0 in term 2.
I20260926 04:03:24.031535 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.031421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36878)
0926 04:03:24.031463 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.031501 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260926 04:03:24.031541 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02076a495c44497996d0923adfd9de22, 1aef1e65b86443d795d601e47f86fbd0; no voters:
W20260926 04:03:24.031666 7686 leader_election.cc:334] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:24.031692 8236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 04:03:24.031729 8236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:24.031754 8236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 2 FOLLOWER]: Advancing to term 3
I20260926 04:03:24.032474 8236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:24.032604 8236 leader_election.cc:288] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 election: Requested vote from peers 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111), 02076a495c44497996d0923adfd9de22 (127.0.200.189:43611)
I20260926 04:03:24.032815 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.032718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36888)
0926 04:03:24.032745 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.032792 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260926 04:03:24.032908 7523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1aef1e65b86443d795d601e47f86fbd0" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "02076a495c44497996d0923adfd9de22"
I20260926 04:03:24.033000 7523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 2 FOLLOWER]: Advancing to term 3
I20260926 04:03:24.033097 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.033022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36904)
0926 04:03:24.033057 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.033077 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260926 04:03:24.033203 7686 leader_election.cc:334] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
I20260926 04:03:24.033746 7523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aef1e65b86443d795d601e47f86fbd0 in term 3.
I20260926 04:03:24.034020 7686 leader_election.cc:302] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02076a495c44497996d0923adfd9de22, 1aef1e65b86443d795d601e47f86fbd0; no voters: 45c0700ef6f440488586dac0ef1f14ee
I20260926 04:03:24.034121 8236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 04:03:24.034260 8236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [term 3 LEADER]: Becoming Leader. State: Replica: 1aef1e65b86443d795d601e47f86fbd0, State: Running, Role: LEADER
I20260926 04:03:24.034358 8236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } }
I20260926 04:03:24.034842 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aef1e65b86443d795d601e47f86fbd0. Latest consensus state: current_term: 3 leader_uuid: "1aef1e65b86443d795d601e47f86fbd0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:24.034952 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:24.035178 8241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:24.035673 8241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:24.035734 8241 catalog_manager.cc:1259] Loaded cluster ID: c3823ba6a43f404abb464f2ff91f1170
I20260926 04:03:24.035786 8241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:24.036031 8241 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:24.036127 8241 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Loaded TSK: 0
I20260926 04:03:24.036244 8241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:24.111367 7700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:24.113009 7700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:24.114193 7523 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [term 3 FOLLOWER]: Refusing update from remote peer 1aef1e65b86443d795d601e47f86fbd0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 04:03:24.114209 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.114077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36918)
0926 04:03:24.114134 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.114173 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 04:03:24.114471 8236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 04:03:24.114751 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.114615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.0.1:36924)
0926 04:03:24.114673 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.114725 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 04:03:24.114845 7686 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 -> Peer 45c0700ef6f440488586dac0ef1f14ee (127.0.200.190:33111): Couldn't send request to peer 45c0700ef6f440488586dac0ef1f14ee. Status: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 04:03:24.115195 8243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aef1e65b86443d795d601e47f86fbd0. Latest consensus state: current_term: 3 leader_uuid: "1aef1e65b86443d795d601e47f86fbd0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:24.115312 8243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:03:24.115720 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1aef1e65b86443d795d601e47f86fbd0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:24.115816 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:24.115969 8244 mvcc.cc:205] Tried to move back new op lower bound from 7333459575248969728 to 7333459574925692928. Current Snapshot: MvccSnapshot[applied={T|T < 7333459575248969728}]
I20260926 04:03:24.115809 8243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1aef1e65b86443d795d601e47f86fbd0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "45c0700ef6f440488586dac0ef1f14ee" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33111 } } peers { permanent_uuid: "02076a495c44497996d0923adfd9de22" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 43611 } attrs { promote: false } } peers { permanent_uuid: "1aef1e65b86443d795d601e47f86fbd0" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 45769 } attrs { promote: false } } }
I20260926 04:03:24.116132 8243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02076a495c44497996d0923adfd9de22 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:03:24.120899 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:39859 (local address 127.0.0.1:55610)
0926 04:03:24.117772 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.117781 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:24.117832 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:24.118294 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:24.118295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:24.118296 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:24.118330 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.118331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.119229 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.119230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.119946 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.119948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.120663 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.120664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.120720 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.120723 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:24.120774 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 04:03:24.120817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":72,"threads_started":1}
I20260926 04:03:24.120903 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.0.200.130:39859)
0926 04:03:24.118167 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:24.118169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:24.118178 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:24.118201 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:24.118202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:24.118212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:24.118386 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:24.118419 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.119171 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.120042 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.120590 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.120665 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.120676 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:24.120792 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 04:03:24.120828 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":59,"threads_started":1}
I20260926 04:03:24.121270 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45461 (local address 127.0.0.1:37558)
0926 04:03:24.117769 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.117781 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:24.117837 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:24.117996 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:24.118004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:24.118006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:24.118060 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.118062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.119150 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.119151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.120312 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.120314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.121058 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.121060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.121132 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.121136 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:24.121165 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 04:03:24.121204 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":40,"thread_start_us":77,"threads_started":1}
I20260926 04:03:24.121487 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:40151 (local address 127.0.0.1:60978)
0926 04:03:24.117742 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.117754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:24.117813 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:24.118139 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:24.118139 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:24.118141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:24.118209 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.118210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.119095 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.119096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.120505 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:24.120506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.121248 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.121249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:24.121310 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.121313 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:24.121364 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 04:03:24.121409 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":30}
I20260926 04:03:24.121558 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.0.200.129:45461)
0926 04:03:24.117785 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:24.117787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:24.117828 (+ 41us) server_negotiation.cc:407] Connection header received
0926 04:03:24.117890 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:24.117891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:24.117901 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:24.117982 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:24.118182 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.119016 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.120378 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.120953 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.121383 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.121397 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:24.121416 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 04:03:24.121467 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260926 04:03:24.121650 7950 tablet_service.cc:1467] Processing CreateTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 (DEFAULT_TABLE table=test-table [id=77764337224343909c1ac836facb2f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:24.121718 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.117679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.0.200.131:40151)
0926 04:03:24.117943 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:24.117945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:24.117954 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:24.117981 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:24.117982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:24.117994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:24.118081 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:24.118292 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.118994 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.120593 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.121166 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.121247 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.121258 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:24.121606 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 04:03:24.121649 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":51,"threads_started":1}
I20260926 04:03:24.121795 7950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:24.122644 7813 tablet_service.cc:1467] Processing CreateTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 (DEFAULT_TABLE table=test-table [id=77764337224343909c1ac836facb2f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:24.122792 7813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:24.123219 8254 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Bootstrap starting.
I20260926 04:03:24.123629 8254 tablet_bootstrap.cc:654] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:24.124248 8254 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: No bootstrap required, opened a new log
I20260926 04:03:24.124307 8254 ts_tablet_manager.cc:1397] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:24.124310 8256 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8: Bootstrap starting.
I20260926 04:03:24.124599 8086 tablet_service.cc:1467] Processing CreateTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 (DEFAULT_TABLE table=test-table [id=77764337224343909c1ac836facb2f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:24.124727 8256 tablet_bootstrap.cc:654] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:24.124749 8086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:24.125304 8256 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8: No bootstrap required, opened a new log
I20260926 04:03:24.125356 8256 ts_tablet_manager.cc:1397] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:24.125509 8256 raft_consensus.cc:348] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.125589 8256 raft_consensus.cc:374] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:24.125617 8256 raft_consensus.cc:729] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b520e255af42ec9730fa5b161178b8, State: Initialized, Role: FOLLOWER
I20260926 04:03:24.125663 8256 consensus_queue.cc:260] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.125823 8256 ts_tablet_manager.cc:1428] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:24.126278 8260 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Bootstrap starting.
I20260926 04:03:24.124524 8254 raft_consensus.cc:348] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.126439 8254 raft_consensus.cc:374] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:24.126470 8254 raft_consensus.cc:729] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce99795d5e014de183abc4b49576e3d2, State: Initialized, Role: FOLLOWER
I20260926 04:03:24.126534 8254 consensus_queue.cc:260] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.126678 7880 heartbeater.cc:503] Master 127.0.200.188:45769 was elected leader, sending a full tablet report...
I20260926 04:03:24.126729 8254 ts_tablet_manager.cc:1428] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:24.127041 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.125984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:35939)
0926 04:03:24.126150 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.126197 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260926 04:03:24.127074 8260 tablet_bootstrap.cc:654] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:24.127182 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.126941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:33163)
0926 04:03:24.127090 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.127130 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":112,"threads_started":1}
I20260926 04:03:24.127480 8017 heartbeater.cc:503] Master 127.0.200.188:45769 was elected leader, sending a full tablet report...
I20260926 04:03:24.127774 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.127502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:45223)
0926 04:03:24.127691 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.127728 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":147,"threads_started":1}
I20260926 04:03:24.127770 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.127679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:35979)
0926 04:03:24.127716 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.127740 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260926 04:03:24.127920 8260 tablet_bootstrap.cc:492] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: No bootstrap required, opened a new log
I20260926 04:03:24.127974 8260 ts_tablet_manager.cc:1397] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:24.128091 8260 raft_consensus.cc:348] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.128166 8260 raft_consensus.cc:374] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:24.128193 8260 raft_consensus.cc:729] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f2ba1281ba43499bf09b79bfa073f1, State: Initialized, Role: FOLLOWER
I20260926 04:03:24.128257 8260 consensus_queue.cc:260] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.128394 8260 ts_tablet_manager.cc:1428] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:24.128835 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.128620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:56681)
0926 04:03:24.128732 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.128800 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":75,"threads_started":1}
I20260926 04:03:24.128868 8153 heartbeater.cc:503] Master 127.0.200.188:45769 was elected leader, sending a full tablet report...
I20260926 04:03:24.129204 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.129105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:60549)
0926 04:03:24.129142 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.129174 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 04:03:24.263267 8265 raft_consensus.cc:482] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:24.263420 8265 raft_consensus.cc:504] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.263878 8265 leader_election.cc:288] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461), ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:24.264046 7970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b43f00f117e4fa1ad0cfd1b8a8e3882" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce99795d5e014de183abc4b49576e3d2" is_pre_election: true
I20260926 04:03:24.264055 7833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b43f00f117e4fa1ad0cfd1b8a8e3882" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b520e255af42ec9730fa5b161178b8" is_pre_election: true
I20260926 04:03:24.264150 7833 raft_consensus.cc:2454] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 0.
I20260926 04:03:24.264150 7970 raft_consensus.cc:2454] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 0.
I20260926 04:03:24.264554 8039 leader_election.cc:302] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f2ba1281ba43499bf09b79bfa073f1, b2b520e255af42ec9730fa5b161178b8; no voters:
I20260926 04:03:24.264832 8265 raft_consensus.cc:2781] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:24.264899 8265 raft_consensus.cc:482] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:24.264933 8265 raft_consensus.cc:3037] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:24.265724 8265 raft_consensus.cc:504] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.265954 8265 leader_election.cc:288] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 election: Requested vote from peers b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461), ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859)
I20260926 04:03:24.266096 7833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b43f00f117e4fa1ad0cfd1b8a8e3882" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b520e255af42ec9730fa5b161178b8"
I20260926 04:03:24.266191 7833 raft_consensus.cc:3037] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:24.266209 7970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b43f00f117e4fa1ad0cfd1b8a8e3882" candidate_uuid: "72f2ba1281ba43499bf09b79bfa073f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce99795d5e014de183abc4b49576e3d2"
I20260926 04:03:24.266314 7970 raft_consensus.cc:3037] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:24.266960 7833 raft_consensus.cc:2454] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 1.
I20260926 04:03:24.267168 8039 leader_election.cc:302] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f2ba1281ba43499bf09b79bfa073f1, b2b520e255af42ec9730fa5b161178b8; no voters:
I20260926 04:03:24.267292 7970 raft_consensus.cc:2454] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f2ba1281ba43499bf09b79bfa073f1 in term 1.
I20260926 04:03:24.267344 8265 raft_consensus.cc:2781] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:24.267628 8265 raft_consensus.cc:686] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 LEADER]: Becoming Leader. State: Replica: 72f2ba1281ba43499bf09b79bfa073f1, State: Running, Role: LEADER
I20260926 04:03:24.268026 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.267848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:55687)
0926 04:03:24.267924 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.267988 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 04:03:24.268020 8265 consensus_queue.cc:237] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } }
I20260926 04:03:24.268953 7699 catalog_manager.cc:5495] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "72f2ba1281ba43499bf09b79bfa073f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72f2ba1281ba43499bf09b79bfa073f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 40151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:24.305059 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.302269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.0.200.131:40151)
0926 04:03:24.302324 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:24.302327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:24.302381 (+ 54us) server_negotiation.cc:407] Connection header received
0926 04:03:24.302457 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:24.302458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:24.302469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:24.302533 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:24.302678 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.303407 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.304229 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:24.304574 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:24.304652 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:24.304764 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:24.304796 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:24.304842 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:24.304891 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 04:03:24.304949 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:24.306080 7833 raft_consensus.cc:1261] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P b2b520e255af42ec9730fa5b161178b8 [term 1 FOLLOWER]: Refusing update from remote peer 72f2ba1281ba43499bf09b79bfa073f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:24.306080 7970 raft_consensus.cc:1261] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Refusing update from remote peer 72f2ba1281ba43499bf09b79bfa073f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:24.306304 8265 consensus_queue.cc:1035] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b520e255af42ec9730fa5b161178b8" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:24.306414 8267 consensus_queue.cc:1035] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce99795d5e014de183abc4b49576e3d2" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:24.307377 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.307105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:33651)
0926 04:03:24.307151 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.307183 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 04:03:24.307535 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.307407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:42403)
0926 04:03:24.307476 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.307505 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 04:03:24.308871 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.308747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:34681)
0926 04:03:24.308799 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.308834 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 04:03:24.309255 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.309153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.131:42475)
0926 04:03:24.309187 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.309229 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 04:03:24.309828 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.309732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.130:49137)
0926 04:03:24.309766 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.309795 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260926 04:03:24.311107 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:24.310974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33111 (local address 127.0.200.129:39421)
0926 04:03:24.311034 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.311077 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:33111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 04:03:24.312198 7700 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:24.312297 7700 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:24.313810 7700 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1aef1e65b86443d795d601e47f86fbd0: Sending DeleteTablet for 3 replicas of tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882
I20260926 04:03:24.314165 7950 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:24 UTC) from {username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:55610
I20260926 04:03:24.314204 7813 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:24 UTC) from {username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:37558
I20260926 04:03:24.314764 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7661
I20260926 04:03:24.316249 8086 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:24 UTC) from {username='oryx', principal='oryx/127.0.200.188@KRBTEST.COM'} at 127.0.0.1:60978
I20260926 04:03:24.316455 8281 tablet_replica.cc:330] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: stopping tablet replica
I20260926 04:03:24.316531 8281 raft_consensus.cc:2229] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:24.316543 8279 tablet_replica.cc:330] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: stopping tablet replica
I20260926 04:03:24.316604 8279 raft_consensus.cc:2229] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:24.316607 8281 raft_consensus.cc:2258] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:24.316738 8279 raft_consensus.cc:2258] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:24.316924 8281 ts_tablet_manager.cc:1905] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:24.317140 8279 ts_tablet_manager.cc:1905] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:24.318688 8281 ts_tablet_manager.cc:1918] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:24.318754 8281 log.cc:1196] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/6b43f00f117e4fa1ad0cfd1b8a8e3882
I20260926 04:03:24.319012 8281 ts_tablet_manager.cc:1939] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P 72f2ba1281ba43499bf09b79bfa073f1: Deleting consensus metadata
I20260926 04:03:24.319060 8279 ts_tablet_manager.cc:1918] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:24.319177 8279 log.cc:1196] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/6b43f00f117e4fa1ad0cfd1b8a8e3882
I20260926 04:03:24.319347 7686 catalog_manager.cc:4841] TS 72f2ba1281ba43499bf09b79bfa073f1 (127.0.200.131:40151): tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 (table test-table [id=77764337224343909c1ac836facb2f67]) successfully deleted
I20260926 04:03:24.319444 8279 ts_tablet_manager.cc:1939] T 6b43f00f117e4fa1ad0cfd1b8a8e3882 P ce99795d5e014de183abc4b49576e3d2: Deleting consensus metadata
I20260926 04:03:24.319792 7686 catalog_manager.cc:4841] TS ce99795d5e014de183abc4b49576e3d2 (127.0.200.130:39859): tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882 (table test-table [id=77764337224343909c1ac836facb2f67]) successfully deleted
I20260926 04:03:24.324108 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7887
W20260926 04:03:24.325835 8249 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:24.323497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45461 (local address 127.0.0.1:37564)
0926 04:03:24.323539 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:24.323551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:24.323623 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:24.325773 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45461: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":20}
W20260926 04:03:24.326061 7684 catalog_manager.cc:4584] TS b2b520e255af42ec9730fa5b161178b8 (127.0.200.129:45461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b43f00f117e4fa1ad0cfd1b8a8e3882: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45461: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 04:03:24.330684 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8023
I20260926 04:03:24.339241 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7379
I20260926 04:03:24.345745 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 7567
2026-09-26T04:03:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6968 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:24 dist-test-slave-j8c1 krb5kdc[8284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:24 dist-test-slave-j8c1 krb5kdc[8284](info): set up 2 sockets
Sep 26 04:03:24 dist-test-slave-j8c1 krb5kdc[8284](info): commencing operation
krb5kdc: starting...
W20260926 04:03:25.387184 802 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab.
Entry for principal oryx/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab.
I20260926 04:03:25.511344 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45967
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab
--principal=oryx/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:45967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:25.595561 8297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:25.595866 8297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:25.595908 8297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:25.597159 8297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:25.597231 8297 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 04:03:25.597254 8297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:25.597277 8297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:25.597307 8297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:25.597337 8297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:25.598941 8297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:45967
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.190.keytab
--principal=oryx/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:25.599246 8297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:25.599493 8297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:25.604887 8305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:25.604902 8297 server_base.cc:1034] running on GCE node
W20260926 04:03:25.604923 8303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:25.604914 8302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:25.605413 8297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:25.605726 8297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:25.606914 8297 hybrid_clock.cc:648] HybridClock initialized: now 1790395405606900 us; error 36 us; skew 500 ppm
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:25.609629 8297 init.cc:377] Logged in from keytab as oryx/127.0.200.190@KRBTEST.COM (short username oryx)
I20260926 04:03:25.610131 8297 webserver.cc:460] Webserver started at http://127.0.200.190:45849/ using document root <none> and password file <none>
I20260926 04:03:25.610391 8297 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:25.610464 8297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:25.610675 8297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:25.611805 8297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "330ab62981a447eda5db212f4b282e27"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "4a893e0fc5348f43c17ba60f6dae4a44"
server_key_iv: "50fdd180eff0b555d8ab2120e9d114c4"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.612176 8297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "330ab62981a447eda5db212f4b282e27"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "4a893e0fc5348f43c17ba60f6dae4a44"
server_key_iv: "50fdd180eff0b555d8ab2120e9d114c4"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.613577 8297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:25.614374 8312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.614630 8297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:25.614738 8297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "330ab62981a447eda5db212f4b282e27"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "4a893e0fc5348f43c17ba60f6dae4a44"
server_key_iv: "50fdd180eff0b555d8ab2120e9d114c4"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.614826 8297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:25.631973 8297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:25.634289 8297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:25.634479 8297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:25.639019 8297 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:45967
I20260926 04:03:25.639113 8374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:45967 every 8 connection(s)
I20260926 04:03:25.639441 8297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:25.640143 8375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:25.642432 8375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Bootstrap starting.
I20260926 04:03:25.643179 8375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:25.643476 8375 log.cc:824] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:25.644225 8375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: No bootstrap required, opened a new log
I20260926 04:03:25.646310 8375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } }
I20260926 04:03:25.646452 8375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:25.646474 8375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330ab62981a447eda5db212f4b282e27, State: Initialized, Role: FOLLOWER
I20260926 04:03:25.646648 8375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } }
I20260926 04:03:25.646737 8375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:25.646790 8375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:25.646845 8375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:25.647091 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8297
I20260926 04:03:25.647200 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:25.647378 802 external_mini_cluster.cc:1428] Setting key 60a31425ef1ea569eb518c254784606e
I20260926 04:03:25.647495 8375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } }
I20260926 04:03:25.647637 8375 leader_election.cc:302] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 330ab62981a447eda5db212f4b282e27; no voters:
I20260926 04:03:25.647818 8375 leader_election.cc:288] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:25.647866 8378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:25.648243 8378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [term 1 LEADER]: Becoming Leader. State: Replica: 330ab62981a447eda5db212f4b282e27, State: Running, Role: LEADER
I20260926 04:03:25.648312 8375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:25.648337 8378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } }
I20260926 04:03:25.648645 8378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 330ab62981a447eda5db212f4b282e27. Latest consensus state: current_term: 1 leader_uuid: "330ab62981a447eda5db212f4b282e27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } } }
I20260926 04:03:25.648754 8378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:25.648648 8382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "330ab62981a447eda5db212f4b282e27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ab62981a447eda5db212f4b282e27" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45967 } } }
I20260926 04:03:25.648900 8382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:25.649080 8386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:25.649626 8386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:25.651316 8386 catalog_manager.cc:1347] Generated new cluster ID: 24d1f34a332c4accacda02251437fc37
I20260926 04:03:25.651371 8386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:25.658399 8386 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:25.659152 8386 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:25.659528 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.647927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.0.200.190:45967)
0926 04:03:25.648108 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:25.648118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:25.648157 (+ 39us) server_negotiation.cc:407] Connection header received
0926 04:03:25.648276 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:25.648283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:25.648338 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:25.649323 (+ 985us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:25.649434 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.650911 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.651935 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.652527 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.652662 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:25.655497 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:25.655501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:25.655501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:25.655512 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:25.657504 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.657950 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.657954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.657954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.658074 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.658317 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.658319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.658319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.658651 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:25.658820 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:25.658927 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 04:03:25.659294 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260926 04:03:25.670323 8386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Generated new TSK 0
I20260926 04:03:25.670591 8386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
I20260926 04:03:25.701682 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45967
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:25.802346 8401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:25.802606 8401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:25.802681 8401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:25.803918 8401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:25.803964 8401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:25.803985 8401 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:25.804003 8401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:25.805559 8401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:25.805819 8401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:25.806089 8401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:25.811247 8407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:25.811264 8406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:25.811292 8409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:25.811404 8401 server_base.cc:1034] running on GCE node
I20260926 04:03:25.811775 8401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:25.812055 8401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:25.813252 8401 hybrid_clock.cc:648] HybridClock initialized: now 1790395405813248 us; error 41 us; skew 500 ppm
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:25.816087 8401 init.cc:377] Logged in from keytab as oryx/127.0.200.129@KRBTEST.COM (short username oryx)
I20260926 04:03:25.816514 8401 webserver.cc:460] Webserver started at http://127.0.200.129:44363/ using document root <none> and password file <none>
I20260926 04:03:25.816766 8401 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:25.816814 8401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:25.816946 8401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:25.818084 8401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "3513473af41a4ded9ea73b1a502fc778"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "1707ebd61c2f5625458d3df65941dba2"
server_key_iv: "2532ca9bce5e00f1e0decf9177ccf353"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.818434 8401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "3513473af41a4ded9ea73b1a502fc778"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "1707ebd61c2f5625458d3df65941dba2"
server_key_iv: "2532ca9bce5e00f1e0decf9177ccf353"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.819928 8401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:25.820720 8416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.820919 8401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:25.821012 8401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "3513473af41a4ded9ea73b1a502fc778"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "1707ebd61c2f5625458d3df65941dba2"
server_key_iv: "2532ca9bce5e00f1e0decf9177ccf353"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.821085 8401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:25.832554 8401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:25.834851 8401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:25.835000 8401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:25.835552 8401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:25.835624 8401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.835673 8401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:25.835703 8401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.842505 8401 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:44749
I20260926 04:03:25.842679 8531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:44749 every 8 connection(s)
I20260926 04:03:25.842943 8401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:25.846496 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.836185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:48169 (local address 127.0.200.190:45967)
0926 04:03:25.836252 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:25.836255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:25.836496 (+ 241us) server_negotiation.cc:407] Connection header received
0926 04:03:25.837206 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:25.837208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:25.837241 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:25.837317 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:25.838247 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.839176 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.840804 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.841205 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.841307 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:25.844085 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:25.844086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:25.844087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:25.844093 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:25.845291 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.845679 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.845679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.845680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.845730 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.845955 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.845956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.845956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.846064 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:25.846117 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:25.846362 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 04:03:25.846431 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 04:03:25.846726 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.836185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.129:48169)
0926 04:03:25.836367 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:25.836403 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:25.837038 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:25.837428 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:25.837433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:25.837832 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:25.838120 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:25.838127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.839346 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.839347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:25.840707 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:25.840710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.841314 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.841314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:25.841473 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:25.842122 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:25.842133 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:25.843997 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:25.845365 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:25.845371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:25.845373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:25.845603 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:25.845799 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:25.845800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:25.845800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:25.845898 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:25.846132 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:25.846136 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:25.846249 (+ 113us) client_negotiation.cc:769] Sending connection context
0926 04:03:25.846333 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 04:03:25.846453 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260926 04:03:25.847234 8532 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45967
I20260926 04:03:25.847325 8532 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:25.847513 8532 heartbeater.cc:511] Master 127.0.200.190:45967 requested a full tablet report, sending...
I20260926 04:03:25.847743 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8401
I20260926 04:03:25.847857 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:25.848042 802 external_mini_cluster.cc:1428] Setting key 3d2dc1fc36057c0f6fa717dc736bf188
I20260926 04:03:25.848071 8328 ts_manager.cc:194] Registered new tserver with Master: 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749)
I20260926 04:03:25.848912 8328 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:48169
I20260926 04:03:25.855934 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.848286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.129:45381)
0926 04:03:25.848619 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:25.848629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:25.850006 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:25.850157 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:25.850158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:25.850339 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:25.850396 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:25.850398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.851275 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.851276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:25.851775 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:25.851777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.852282 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.852283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:25.852338 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:25.852846 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:25.852851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:25.853167 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:25.854900 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:25.854901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:25.854901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:25.855075 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:25.855297 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:25.855297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:25.855298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:25.855348 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:25.855643 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:25.855644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:25.855687 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 04:03:25.855765 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 04:03:25.855852 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":69,"threads_started":1}
I20260926 04:03:25.856004 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.848355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:45381 (local address 127.0.200.190:45967)
0926 04:03:25.848560 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:25.848562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:25.848667 (+ 105us) server_negotiation.cc:407] Connection header received
0926 04:03:25.850092 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:25.850093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:25.850105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:25.850154 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:25.850471 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.851176 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.851863 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:25.852179 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:25.852258 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:25.853260 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:25.853261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:25.853262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:25.853269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:25.854821 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.855151 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.855153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.855153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.855229 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:25.855411 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:25.855412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:25.855413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:25.855547 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:25.855625 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:25.855816 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 04:03:25.855890 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
WARNING: no policy specified for oryx/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
I20260926 04:03:25.875801 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45967
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:25.957877 8541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:25.958144 8541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:25.958197 8541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:25.959470 8541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:25.959547 8541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:25.959585 8541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:25.959617 8541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:25.961200 8541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:25.961476 8541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:25.961724 8541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:25.967056 8547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:25.967056 8549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:25.967060 8546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:25.967244 8541 server_base.cc:1034] running on GCE node
I20260926 04:03:25.967546 8541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:25.967842 8541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:25.969049 8541 hybrid_clock.cc:648] HybridClock initialized: now 1790395405969045 us; error 39 us; skew 500 ppm
Sep 26 04:03:25 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:25.971875 8541 init.cc:377] Logged in from keytab as oryx/127.0.200.130@KRBTEST.COM (short username oryx)
I20260926 04:03:25.972337 8541 webserver.cc:460] Webserver started at http://127.0.200.130:36785/ using document root <none> and password file <none>
I20260926 04:03:25.972563 8541 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:25.972636 8541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:25.972784 8541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:25.973820 8541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "887a61786275882b8627072255bb4d53"
server_key_iv: "175972a44ce82d31211f8a7271554b9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.974180 8541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "887a61786275882b8627072255bb4d53"
server_key_iv: "175972a44ce82d31211f8a7271554b9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.975649 8541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:25.976533 8556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.976711 8541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:25.976845 8541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
format_stamp: "Formatted at 2026-09-26 04:03:25 on dist-test-slave-j8c1"
server_key: "887a61786275882b8627072255bb4d53"
server_key_iv: "175972a44ce82d31211f8a7271554b9a"
server_key_version: "encryptionkey@0"
I20260926 04:03:25.976928 8541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:25.994688 8541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:25.997135 8541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:25.997321 8541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:25.998039 8541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:25.998100 8541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:25.998143 8541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:25.998171 8541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:26.005275 8541 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:33049
I20260926 04:03:26.005344 8671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:33049 every 8 connection(s)
I20260926 04:03:26.005650 8541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:26.009053 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.998631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43205 (local address 127.0.200.190:45967)
0926 04:03:25.998716 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:25.998719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:25.998869 (+ 150us) server_negotiation.cc:407] Connection header received
0926 04:03:25.999369 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:25.999371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:25.999384 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:25.999470 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.000176 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.001063 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.002815 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.003018 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.003108 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.005805 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.005807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.005807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.005833 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.007415 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.008004 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.008006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.008006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.008081 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.008364 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.008365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.008365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.008486 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.008545 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.008905 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.008987 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:03:26.009157 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:25.998552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.130:43205)
0926 04:03:25.998727 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:25.998767 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:25.999271 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:25.999560 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:25.999565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:25.999880 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:26.000074 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.000079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.001172 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.001174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.002713 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.002718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.003124 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.003125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.003267 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.004051 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:26.004059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:26.005662 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:26.007551 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.007559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.007561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.007909 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.008171 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.008171 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.008172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.008282 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.008569 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:26.008575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:26.008744 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.008846 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.008982 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260926 04:03:26.009755 8672 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45967
I20260926 04:03:26.009850 8672 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:26.010035 8672 heartbeater.cc:511] Master 127.0.200.190:45967 requested a full tablet report, sending...
I20260926 04:03:26.010427 8328 ts_manager.cc:194] Registered new tserver with Master: 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049)
I20260926 04:03:26.010898 8328 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:43205
I20260926 04:03:26.011238 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8541
I20260926 04:03:26.011340 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:26.011543 802 external_mini_cluster.cc:1428] Setting key a2504b52485fa201ac0d2d087f916779
I20260926 04:03:26.018478 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.011070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:57983 (local address 127.0.200.190:45967)
0926 04:03:26.011111 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.011113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.011927 (+ 814us) server_negotiation.cc:407] Connection header received
0926 04:03:26.011956 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.011957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.011966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.012030 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.012414 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.012900 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.013906 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.014219 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.014301 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.015527 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.015528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.015528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.015534 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.017163 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.017616 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.017617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.017618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.017670 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.017912 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.017913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.017913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.018057 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.018114 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.018346 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.018410 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:26.018575 8678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.011715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.130:57983)
0926 04:03:26.011845 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.011855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.011922 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.012031 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.012031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.012252 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:26.012336 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.012338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.013003 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.013004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.013805 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.013807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.014326 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.014327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.014382 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.014939 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:26.014945 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:26.015415 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:26.017267 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.017268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.017268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.017538 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.017770 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.017770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.017771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.017847 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.018131 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:26.018132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:26.018194 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.018297 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.018418 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for oryx/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
I20260926 04:03:26.042217 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45967
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:26.126070 8681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:26.126305 8681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:26.126334 8681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:26.127787 8681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:26.127840 8681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:26.127863 8681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 04:03:26.127882 8681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:26.129361 8681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.200.190:45967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:26.129563 8681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:26.129750 8681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:26.135100 8689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:26.135149 8687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:26.135100 8686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:26.135385 8681 server_base.cc:1034] running on GCE node
I20260926 04:03:26.135645 8681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:26.135957 8681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:26.137104 8681 hybrid_clock.cc:648] HybridClock initialized: now 1790395406137080 us; error 41 us; skew 500 ppm
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395406, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:26.140039 8681 init.cc:377] Logged in from keytab as oryx/127.0.200.131@KRBTEST.COM (short username oryx)
I20260926 04:03:26.140537 8681 webserver.cc:460] Webserver started at http://127.0.200.131:46877/ using document root <none> and password file <none>
I20260926 04:03:26.140777 8681 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:26.140847 8681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:26.141000 8681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:26.142056 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "f51bbb4c6c27417a813a7dfbf5e761b1"
format_stamp: "Formatted at 2026-09-26 04:03:26 on dist-test-slave-j8c1"
server_key: "ff21a29ab667137b764a4e36b7ec7c3d"
server_key_iv: "4cb61fca2db67a1d83db3e4ea8a6e601"
server_key_version: "encryptionkey@0"
I20260926 04:03:26.142426 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "f51bbb4c6c27417a813a7dfbf5e761b1"
format_stamp: "Formatted at 2026-09-26 04:03:26 on dist-test-slave-j8c1"
server_key: "ff21a29ab667137b764a4e36b7ec7c3d"
server_key_iv: "4cb61fca2db67a1d83db3e4ea8a6e601"
server_key_version: "encryptionkey@0"
I20260926 04:03:26.143805 8681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:26.144706 8696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:26.144944 8681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:26.145052 8681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "f51bbb4c6c27417a813a7dfbf5e761b1"
format_stamp: "Formatted at 2026-09-26 04:03:26 on dist-test-slave-j8c1"
server_key: "ff21a29ab667137b764a4e36b7ec7c3d"
server_key_iv: "4cb61fca2db67a1d83db3e4ea8a6e601"
server_key_version: "encryptionkey@0"
I20260926 04:03:26.145143 8681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:26.157513 8681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:26.160108 8681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:26.160356 8681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:26.161053 8681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:26.161105 8681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:26.161144 8681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:26.161157 8681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395406, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:26.168612 8681 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:35707
I20260926 04:03:26.169013 8811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:35707 every 8 connection(s)
I20260926 04:03:26.169091 8681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:26.172083 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.161497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:50377 (local address 127.0.200.190:45967)
0926 04:03:26.161575 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.161577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.161794 (+ 217us) server_negotiation.cc:407] Connection header received
0926 04:03:26.162323 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.162325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.162338 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.162419 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.163192 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.163834 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.165089 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.165321 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.165416 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.168779 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.168780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.168781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.168790 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.170538 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.171079 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.171080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.171081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.171151 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.171411 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.171412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.171413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.171545 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.171624 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.171908 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.171993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:03:26.172290 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.161493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.131:50377)
0926 04:03:26.161648 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.161692 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.162221 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.162501 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.162505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.162846 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:26.163062 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.163067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.163938 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.163940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.164988 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.164992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.165426 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.165427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.165570 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.166163 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:26.166172 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:26.168620 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:26.170666 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.170673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.170675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.170984 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.171228 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.171229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.171230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.171343 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.171614 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:26.171620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:26.171769 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.171874 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.172040 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260926 04:03:26.172828 8812 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45967
I20260926 04:03:26.172907 8812 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:26.173112 8812 heartbeater.cc:511] Master 127.0.200.190:45967 requested a full tablet report, sending...
I20260926 04:03:26.173475 8329 ts_manager.cc:194] Registered new tserver with Master: f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707)
I20260926 04:03:26.173889 8329 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:50377
I20260926 04:03:26.177629 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8681
I20260926 04:03:26.177721 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:26.177906 802 external_mini_cluster.cc:1428] Setting key d50b88b09c4d39515c60641c9dc65617
I20260926 04:03:26.178956 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:03:26.180363 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.174316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:46675 (local address 127.0.200.190:45967)
0926 04:03:26.174366 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.174368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.174383 (+ 15us) server_negotiation.cc:407] Connection header received
0926 04:03:26.174412 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.174412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.174420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.174471 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.174907 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.175425 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.176126 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.176316 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.176392 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.177379 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.177380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.177381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.177387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.179142 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.179537 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.179538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.179538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.179632 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.179858 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.179859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.179859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.179968 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.180076 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.180233 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.180301 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 04:03:26.180480 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.174138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.200.131:46675)
0926 04:03:26.174283 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.174294 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.174342 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.174496 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.174497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.174732 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:26.174807 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.174809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.175515 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.175515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.176055 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.176056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.176402 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.176403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.176448 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.176963 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:26.176968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:26.177281 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:26.179211 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.179212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.179212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.179440 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.179714 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.179715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.179715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.179795 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.180069 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:26.180070 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:26.180122 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.180205 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.180324 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 04:03:26.200289 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:26.194823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.0.200.190:45967)
0926 04:03:26.194895 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.194897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.194910 (+ 13us) server_negotiation.cc:407] Connection header received
0926 04:03:26.194938 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.194939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.194949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.195008 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.195575 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.196092 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.196807 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.197018 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.197076 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.200195 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40036: BlockingRecv error: recv got EOF from 127.0.0.1:40036 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:26.207930 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.201153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.0.200.190:45967)
0926 04:03:26.201201 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.201204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.201263 (+ 59us) server_negotiation.cc:407] Connection header received
0926 04:03:26.201302 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.201303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.201311 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.201366 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.201880 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.202351 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.203146 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.203329 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.203385 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.205475 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.205476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.205477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.205484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.206833 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.207161 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.207162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.207162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.207214 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.207391 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.207392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.207392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.207497 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.207569 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.207741 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.207804 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:03:26.209632 8329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:26.211508 8329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:26.218060 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.213934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44749 (local address 127.0.0.1:58844)
0926 04:03:26.214113 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.214152 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.214253 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.214571 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.214576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.214578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.214650 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.214651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.215663 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.215664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.216964 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.216966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.217736 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.217824 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.217838 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.217907 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.217964 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":60,"threads_started":1}
I20260926 04:03:26.218067 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.214040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.0.200.129:44749)
0926 04:03:26.214389 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.214398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.214414 (+ 16us) server_negotiation.cc:407] Connection header received
0926 04:03:26.214452 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.214454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.214468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.214567 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.214760 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.215564 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217069 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.217632 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217732 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.217812 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.217938 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.217973 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":72,"threads_started":1}
I20260926 04:03:26.218194 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.213944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.0.1:35450)
0926 04:03:26.214352 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.214361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.214399 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.214900 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.214900 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.214902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.214935 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.214937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.216106 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.216107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.216969 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.216971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.217957 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.218021 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.218026 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.218067 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.218108 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":85,"threads_started":1}
I20260926 04:03:26.218518 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.214164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.0.200.131:35707)
0926 04:03:26.214694 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.214698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.214708 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:26.214746 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.214747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.214768 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.214840 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.215006 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.216011 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217051 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.217793 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217862 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.218340 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.218369 (+ 29us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.218417 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":101,"threads_started":1}
I20260926 04:03:26.218613 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.213934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:33049 (local address 127.0.0.1:60468)
0926 04:03:26.214196 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.214207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.214262 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.214543 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.214601 (+ 58us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.214604 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.214661 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.214664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.215966 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.215967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.217090 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.217092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.218034 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.218035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.218090 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.218093 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.218464 (+ 371us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.218508 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":29,"thread_start_us":82,"threads_started":1}
I20260926 04:03:26.218878 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.214172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.0.200.130:33049)
0926 04:03:26.214381 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.214386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.214398 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:26.214436 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.214438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.214450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.214541 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.214763 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.215878 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217198 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.217882 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.217942 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.218006 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.218695 (+ 689us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.218756 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
I20260926 04:03:26.218850 8466 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (DEFAULT_TABLE table=test-table [id=d05724734b564be6a25ed385396c5670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.219238 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.219467 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (DEFAULT_TABLE table=test-table [id=d05724734b564be6a25ed385396c5670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.219785 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.222661 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (DEFAULT_TABLE table=test-table [id=d05724734b564be6a25ed385396c5670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.223019 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.223454 8841 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Bootstrap starting.
I20260926 04:03:26.223596 8840 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Bootstrap starting.
I20260926 04:03:26.224175 8840 tablet_bootstrap.cc:654] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.224315 8841 tablet_bootstrap.cc:654] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.224581 8840 log.cc:824] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:26.224733 8841 log.cc:824] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:26.225451 8841 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: No bootstrap required, opened a new log
I20260926 04:03:26.225452 8840 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: No bootstrap required, opened a new log
I20260926 04:03:26.225538 8841 ts_tablet_manager.cc:1397] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:26.225539 8840 ts_tablet_manager.cc:1397] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:26.227097 8844 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Bootstrap starting.
I20260926 04:03:26.227228 8841 raft_consensus.cc:348] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.227391 8841 raft_consensus.cc:374] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.227438 8841 raft_consensus.cc:729] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.227573 8841 consensus_queue.cc:260] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.227869 8841 ts_tablet_manager.cc:1428] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 04:03:26.227989 8672 heartbeater.cc:503] Master 127.0.200.190:45967 was elected leader, sending a full tablet report...
I20260926 04:03:26.228080 8840 raft_consensus.cc:348] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.228235 8840 raft_consensus.cc:374] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.228260 8840 raft_consensus.cc:729] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.228368 8844 tablet_bootstrap.cc:654] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.228404 8840 consensus_queue.cc:260] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.228670 8840 ts_tablet_manager.cc:1428] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:03:26.228705 8532 heartbeater.cc:503] Master 127.0.200.190:45967 was elected leader, sending a full tablet report...
I20260926 04:03:26.228715 8844 log.cc:824] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:26.229570 8844 tablet_bootstrap.cc:492] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: No bootstrap required, opened a new log
I20260926 04:03:26.229651 8844 ts_tablet_manager.cc:1397] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 04:03:26.231233 8844 raft_consensus.cc:348] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.231386 8844 raft_consensus.cc:374] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.231410 8844 raft_consensus.cc:729] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.231487 8844 consensus_queue.cc:260] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.231688 8844 ts_tablet_manager.cc:1428] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:26.231775 8812 heartbeater.cc:503] Master 127.0.200.190:45967 was elected leader, sending a full tablet report...
W20260926 04:03:26.256323 8673 tablet.cc:2369] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:03:26.343775 8533 tablet.cc:2369] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:26.370225 8848 raft_consensus.cc:482] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:26.370412 8848 raft_consensus.cc:504] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.370865 8848 leader_election.cc:288] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049), 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749)
I20260926 04:03:26.373889 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.371231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:50555 (local address 127.0.200.130:33049)
0926 04:03:26.371278 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.371280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.371289 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:26.371316 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.371317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.371325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.371400 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.371559 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.372048 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373099 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.373473 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373559 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.373568 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.373768 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.373804 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:03:26.373903 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.371101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:33049 (local address 127.0.200.131:50555)
0926 04:03:26.371162 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.371177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.371245 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.371404 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.371405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.371406 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.371481 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.371483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.372143 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.372144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.373015 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.373018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.373557 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.373667 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.373670 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.373758 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.373804 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 04:03:26.374048 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.371101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44749 (local address 127.0.200.131:42549)
0926 04:03:26.371337 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.371346 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.371424 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.371577 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.371578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.371579 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.371654 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.371655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.372296 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.372297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.373208 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.373211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.373755 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.373832 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.373836 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.373910 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.373954 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":70,"thread_start_us":101,"threads_started":1}
I20260926 04:03:26.374042 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.371218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:42549 (local address 127.0.200.129:44749)
0926 04:03:26.371266 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.371268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.371411 (+ 143us) server_negotiation.cc:407] Connection header received
0926 04:03:26.371492 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.371493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.371501 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.371575 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.371756 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.372210 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373293 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.373650 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.373752 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.373760 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.373938 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.373978 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:03:26.374251 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1c35fb625d43978e4f4f70d5cd0f5a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" is_pre_election: true
I20260926 04:03:26.374413 8626 raft_consensus.cc:2454] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:26.374418 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1c35fb625d43978e4f4f70d5cd0f5a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778" is_pre_election: true
I20260926 04:03:26.374624 8486 raft_consensus.cc:2454] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:26.374651 8697 leader_election.cc:302] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.374783 8848 raft_consensus.cc:2781] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:26.374852 8848 raft_consensus.cc:482] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:26.374893 8848 raft_consensus.cc:3037] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.375983 8848 raft_consensus.cc:504] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.376140 8848 leader_election.cc:288] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Requested vote from peers 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049), 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749)
I20260926 04:03:26.376319 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1c35fb625d43978e4f4f70d5cd0f5a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
I20260926 04:03:26.376410 8626 raft_consensus.cc:3037] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.377180 8626 raft_consensus.cc:2454] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:26.377280 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1c35fb625d43978e4f4f70d5cd0f5a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778"
I20260926 04:03:26.377377 8697 leader_election.cc:302] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.377386 8486 raft_consensus.cc:3037] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.377472 8848 raft_consensus.cc:2781] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:26.377655 8848 raft_consensus.cc:686] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 LEADER]: Becoming Leader. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Running, Role: LEADER
I20260926 04:03:26.377871 8848 consensus_queue.cc:237] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.378224 8486 raft_consensus.cc:2454] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:26.378880 8329 catalog_manager.cc:5495] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:26.403062 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.400232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.0.200.131:35707)
0926 04:03:26.400288 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.400290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.400298 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:26.400332 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.400333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.400351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.400446 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:26.400625 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.401212 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.402151 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.402367 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.402451 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.402658 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.402775 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.402926 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.402945 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.402972 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:03:26.404942 8626 raft_consensus.cc:1261] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.405232 8486 raft_consensus.cc:1261] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.405334 8848 consensus_queue.cc:1035] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.405546 8850 consensus_queue.cc:1035] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.408344 8854 mvcc.cc:205] Tried to move back new op lower bound from 7333459584631762944 to 7333459584524382208. Current Snapshot: MvccSnapshot[applied={T|T < 7333459584631762944}]
I20260926 04:03:26.408416 8855 mvcc.cc:205] Tried to move back new op lower bound from 7333459584631762944 to 7333459584524382208. Current Snapshot: MvccSnapshot[applied={T|T < 7333459584631762944}]
I20260926 04:03:26.410306 8328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:26.410462 8328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:26.411293 8328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Sending DeleteTablet for 3 replicas of tablet 4a1c35fb625d43978e4f4f70d5cd0f5a
I20260926 04:03:26.411651 8466 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:26 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260926 04:03:26.411655 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:26 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:60468
I20260926 04:03:26.411919 8863 tablet_replica.cc:330] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: stopping tablet replica
I20260926 04:03:26.411896 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1c35fb625d43978e4f4f70d5cd0f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:26 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35450
I20260926 04:03:26.412020 8863 raft_consensus.cc:2229] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:26.412125 8863 raft_consensus.cc:2258] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:26.412088 8328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:26.412228 8328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:26.412143 8865 tablet_replica.cc:330] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: stopping tablet replica
I20260926 04:03:26.412762 8864 tablet_replica.cc:330] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: stopping tablet replica
I20260926 04:03:26.412866 8864 raft_consensus.cc:2229] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:26.412806 8863 ts_tablet_manager.cc:1905] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:26.412989 8864 raft_consensus.cc:2258] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:26.413389 8864 ts_tablet_manager.cc:1905] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:26.413933 8466 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d (DEFAULT_TABLE table=test-table [id=db3859303fba42ae92040bc2a1985735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.414324 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d (DEFAULT_TABLE table=test-table [id=db3859303fba42ae92040bc2a1985735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.414430 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec75bfba08643698d3d2dc7dc18d66d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.414444 8865 raft_consensus.cc:2229] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:26.414592 8865 raft_consensus.cc:2258] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:26.414990 8865 ts_tablet_manager.cc:1905] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:26.415251 8864 ts_tablet_manager.cc:1918] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:26.415362 8864 log.cc:1196] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/4a1c35fb625d43978e4f4f70d5cd0f5a
I20260926 04:03:26.415673 8864 ts_tablet_manager.cc:1939] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting consensus metadata
I20260926 04:03:26.415760 8841 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Bootstrap starting.
I20260926 04:03:26.416186 8841 tablet_bootstrap.cc:654] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.414089 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d (DEFAULT_TABLE table=test-table [id=db3859303fba42ae92040bc2a1985735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:26.416778 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec75bfba08643698d3d2dc7dc18d66d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.416788 8865 ts_tablet_manager.cc:1918] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:26.416898 8865 log.cc:1196] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/4a1c35fb625d43978e4f4f70d5cd0f5a
I20260926 04:03:26.417176 8865 ts_tablet_manager.cc:1939] T 4a1c35fb625d43978e4f4f70d5cd0f5a P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting consensus metadata
I20260926 04:03:26.417646 8316 catalog_manager.cc:4841] TS f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707): tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (table test-table [id=d05724734b564be6a25ed385396c5670]) successfully deleted
I20260926 04:03:26.417987 8844 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Bootstrap starting.
I20260926 04:03:26.418100 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec75bfba08643698d3d2dc7dc18d66d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.418435 8844 tablet_bootstrap.cc:654] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.419158 8844 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: No bootstrap required, opened a new log
I20260926 04:03:26.419217 8844 ts_tablet_manager.cc:1397] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:26.419325 8840 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Bootstrap starting.
I20260926 04:03:26.419385 8844 raft_consensus.cc:348] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.419461 8844 raft_consensus.cc:374] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.419481 8844 raft_consensus.cc:729] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.419495 8313 catalog_manager.cc:4841] TS 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049): tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (table test-table [id=d05724734b564be6a25ed385396c5670]) successfully deleted
I20260926 04:03:26.419531 8844 consensus_queue.cc:260] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.419620 8844 ts_tablet_manager.cc:1428] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260926 04:03:26.419706 8840 tablet_bootstrap.cc:654] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Neither blocks nor log segments found. Creating new log.
W20260926 04:03:26.419881 8813 tablet.cc:2369] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:26.419975 8841 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: No bootstrap required, opened a new log
I20260926 04:03:26.420022 8841 ts_tablet_manager.cc:1397] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 04:03:26.420174 8841 raft_consensus.cc:348] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.420256 8841 raft_consensus.cc:374] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.420284 8841 raft_consensus.cc:729] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.420333 8841 consensus_queue.cc:260] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.420430 8841 ts_tablet_manager.cc:1428] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:26.421170 8863 ts_tablet_manager.cc:1918] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:26.421275 8863 log.cc:1196] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/4a1c35fb625d43978e4f4f70d5cd0f5a
I20260926 04:03:26.421555 8863 ts_tablet_manager.cc:1939] T 4a1c35fb625d43978e4f4f70d5cd0f5a P 3513473af41a4ded9ea73b1a502fc778: Deleting consensus metadata
I20260926 04:03:26.422197 8314 catalog_manager.cc:4841] TS 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749): tablet 4a1c35fb625d43978e4f4f70d5cd0f5a (table test-table [id=d05724734b564be6a25ed385396c5670]) successfully deleted
I20260926 04:03:26.427037 8840 tablet_bootstrap.cc:492] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: No bootstrap required, opened a new log
I20260926 04:03:26.427104 8840 ts_tablet_manager.cc:1397] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260926 04:03:26.427316 8840 raft_consensus.cc:348] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.427412 8840 raft_consensus.cc:374] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.427438 8840 raft_consensus.cc:729] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.427490 8840 consensus_queue.cc:260] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.427870 8840 ts_tablet_manager.cc:1428] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 04:03:26.604416 8845 raft_consensus.cc:482] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:26.604539 8845 raft_consensus.cc:504] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.604926 8845 leader_election.cc:288] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707)
I20260926 04:03:26.609766 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.605329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60515 (local address 127.0.200.131:35707)
0926 04:03:26.605429 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.605431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.605443 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:26.605474 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.605475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.605487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.605573 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.605780 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.606535 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.608301 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.609087 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.609224 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.609239 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.609602 (+ 363us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.609682 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 04:03:26.609766 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.605470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:47723 (local address 127.0.200.129:44749)
0926 04:03:26.605524 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.605526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.605534 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:26.605598 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.605600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.605609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.605670 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:26.605898 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.606631 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.608465 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.609173 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.609259 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.609270 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.609607 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.609682 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 04:03:26.609779 8869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.605160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.200.130:60515)
0926 04:03:26.605294 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.605305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.605398 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.605587 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.605589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.605591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.605695 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.605697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.606636 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.606637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.608132 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.608135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.609305 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.609307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.609423 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.609433 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.609521 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.609610 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":69,"thread_start_us":67,"threads_started":1}
I20260926 04:03:26.609781 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.605386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44749 (local address 127.0.200.130:47723)
0926 04:03:26.605490 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.605498 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.605541 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.605737 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.605738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.605740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:26.605789 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.605790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.606717 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.606717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.608132 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.608135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.609306 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.609307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.609423 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.609434 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.609521 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.609614 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":85,"thread_start_us":62,"threads_started":1}
I20260926 04:03:26.610212 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec75bfba08643698d3d2dc7dc18d66d" candidate_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" is_pre_election: true
I20260926 04:03:26.610414 8766 raft_consensus.cc:2454] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5977665e9a4d4174b1d8c7fba25f32a1 in term 0.
I20260926 04:03:26.610664 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec75bfba08643698d3d2dc7dc18d66d" candidate_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778" is_pre_election: true
I20260926 04:03:26.610657 8560 leader_election.cc:302] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.610769 8486 raft_consensus.cc:2454] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5977665e9a4d4174b1d8c7fba25f32a1 in term 0.
I20260926 04:03:26.610874 8845 raft_consensus.cc:2781] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:26.610927 8845 raft_consensus.cc:482] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:26.610950 8845 raft_consensus.cc:3037] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.611670 8845 raft_consensus.cc:504] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.611835 8845 leader_election.cc:288] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [CANDIDATE]: Term 1 election: Requested vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707)
I20260926 04:03:26.611982 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec75bfba08643698d3d2dc7dc18d66d" candidate_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778"
I20260926 04:03:26.612030 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec75bfba08643698d3d2dc7dc18d66d" candidate_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1"
I20260926 04:03:26.612089 8766 raft_consensus.cc:3037] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.612092 8486 raft_consensus.cc:3037] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.612792 8766 raft_consensus.cc:2454] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5977665e9a4d4174b1d8c7fba25f32a1 in term 1.
I20260926 04:03:26.612962 8560 leader_election.cc:302] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.613075 8845 raft_consensus.cc:2781] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:26.613219 8845 raft_consensus.cc:686] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Running, Role: LEADER
I20260926 04:03:26.613334 8486 raft_consensus.cc:2454] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5977665e9a4d4174b1d8c7fba25f32a1 in term 1.
I20260926 04:03:26.613333 8845 consensus_queue.cc:237] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.614122 8328 catalog_manager.cc:5495] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130). New cstate: current_term: 1 leader_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } health_report { overall_health: UNKNOWN } } }
Sep 26 04:03:26 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395405, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.190@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:26.674589 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.667403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45967 (local address 127.0.0.1:39904)
0926 04:03:26.667611 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.667628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.667732 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.667886 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.667887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.668241 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:26.668288 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.668291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.668955 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.668956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.669476 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.669478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.669852 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.669852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.669910 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.670460 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:26.670465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:26.671906 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:26.673520 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.673526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.673528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.673782 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.673972 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:26.673972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:26.673972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:26.674027 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:26.674256 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:26.674261 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:26.674303 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.674359 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.674451 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":118,"threads_started":1}
I20260926 04:03:26.674763 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.667467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.0.200.190:45967)
0926 04:03:26.667573 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.667575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.667743 (+ 168us) server_negotiation.cc:407] Connection header received
0926 04:03:26.667795 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.667796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.667806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.667882 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:26.668378 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.668870 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.669576 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.669763 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.669849 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.672012 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:26.672014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:26.672014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:26.672020 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:26.673399 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.673856 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.673857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.673857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.673908 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:26.674082 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:26.674083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:26.674083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:26.674189 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:26.674242 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.674388 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.674494 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":73}
I20260926 04:03:26.676649 8328 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:39904:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 04:03:26.676923 8328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:26.678406 8466 tablet_service.cc:1467] Processing CreateTablet for tablet e73145da9f004f7c80b21914566fe55a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4e004364da74dcbb6cfd969e971833c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:26.678438 8746 tablet_service.cc:1467] Processing CreateTablet for tablet e73145da9f004f7c80b21914566fe55a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4e004364da74dcbb6cfd969e971833c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:26.678599 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73145da9f004f7c80b21914566fe55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.678612 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73145da9f004f7c80b21914566fe55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.679791 8844 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: Bootstrap starting.
I20260926 04:03:26.679942 8840 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: Bootstrap starting.
I20260926 04:03:26.680042 8606 tablet_service.cc:1467] Processing CreateTablet for tablet e73145da9f004f7c80b21914566fe55a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4e004364da74dcbb6cfd969e971833c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 04:03:26.680155 8844 tablet_bootstrap.cc:654] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.680182 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e73145da9f004f7c80b21914566fe55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:26.680346 8840 tablet_bootstrap.cc:654] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.681018 8840 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: No bootstrap required, opened a new log
I20260926 04:03:26.681085 8840 ts_tablet_manager.cc:1397] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:26.681221 8840 raft_consensus.cc:348] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.681298 8840 raft_consensus.cc:374] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.681317 8840 raft_consensus.cc:729] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.681401 8840 consensus_queue.cc:260] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.681516 8840 ts_tablet_manager.cc:1428] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:26.681644 8846 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.681787 8846 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.681804 8841 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: Bootstrap starting.
I20260926 04:03:26.682101 8844 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: No bootstrap required, opened a new log
I20260926 04:03:26.682163 8844 ts_tablet_manager.cc:1397] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:26.682332 8841 tablet_bootstrap.cc:654] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:26.682304 8844 raft_consensus.cc:348] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.682370 8844 raft_consensus.cc:374] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.682394 8844 raft_consensus.cc:729] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.682440 8844 consensus_queue.cc:260] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.682533 8844 ts_tablet_manager.cc:1428] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:26.682587 8858 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.682672 8858 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.683308 8841 tablet_bootstrap.cc:492] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: No bootstrap required, opened a new log
I20260926 04:03:26.683355 8841 ts_tablet_manager.cc:1397] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:03:26.683475 8841 raft_consensus.cc:348] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.683530 8841 raft_consensus.cc:374] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:26.683545 8841 raft_consensus.cc:729] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Initialized, Role: FOLLOWER
I20260926 04:03:26.683606 8841 consensus_queue.cc:260] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.683737 8841 ts_tablet_manager.cc:1428] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:26.683772 8845 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.683862 8845 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.918704 8858 raft_consensus.cc:482] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:26.918841 8858 raft_consensus.cc:504] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.919066 8858 leader_election.cc:288] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049)
I20260926 04:03:26.919296 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73145da9f004f7c80b21914566fe55a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" is_pre_election: true
I20260926 04:03:26.919384 8626 raft_consensus.cc:2454] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:26.919473 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73145da9f004f7c80b21914566fe55a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778" is_pre_election: true
I20260926 04:03:26.919572 8486 raft_consensus.cc:2454] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:26.919600 8697 leader_election.cc:302] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.919741 8858 raft_consensus.cc:2781] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:26.919781 8858 raft_consensus.cc:482] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:26.919816 8858 raft_consensus.cc:3037] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.920568 8858 raft_consensus.cc:504] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.920724 8858 leader_election.cc:288] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Requested vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049)
I20260926 04:03:26.920920 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73145da9f004f7c80b21914566fe55a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778"
I20260926 04:03:26.920920 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73145da9f004f7c80b21914566fe55a" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
I20260926 04:03:26.921016 8626 raft_consensus.cc:3037] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.921016 8486 raft_consensus.cc:3037] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:26.921788 8486 raft_consensus.cc:2454] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:26.921788 8626 raft_consensus.cc:2454] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:26.922137 8697 leader_election.cc:302] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5977665e9a4d4174b1d8c7fba25f32a1, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:26.922443 8858 raft_consensus.cc:2781] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:26.922637 8858 raft_consensus.cc:686] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 LEADER]: Becoming Leader. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Running, Role: LEADER
I20260926 04:03:26.922720 8858 consensus_queue.cc:237] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:26.923009 8882 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: TxnStatusTablet state changed. Reason: New leader f51bbb4c6c27417a813a7dfbf5e761b1. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.923139 8882 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:26.923287 8884 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 04:03:26.923421 8884 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 04:03:26.923359 8327 catalog_manager.cc:5495] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:26.968466 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.965245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.0.1:34160)
0926 04:03:26.965319 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.965331 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.965384 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.965672 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.965673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.965674 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:26.965720 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.965722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.966657 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.966657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.967254 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.967256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.967802 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.967804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.967889 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.967919 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.968283 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.968293 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.968345 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.968397 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:03:26.968477 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.965466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.0.200.131:35707)
0926 04:03:26.965540 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.965542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.965552 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:26.965579 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.965580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.965591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.965653 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:26.965790 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.966573 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.967335 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.967640 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.967726 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.968147 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.968184 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.968241 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.968374 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.968396 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 04:03:26.969758 8486 raft_consensus.cc:1261] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.969779 8626 raft_consensus.cc:1261] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.969982 8882 consensus_queue.cc:1035] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.970093 8858 consensus_queue.cc:1035] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.970463 8846 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: TxnStatusTablet state changed. Reason: New leader f51bbb4c6c27417a813a7dfbf5e761b1. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.970567 8846 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.970846 8845 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: TxnStatusTablet state changed. Reason: New leader f51bbb4c6c27417a813a7dfbf5e761b1. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.970932 8845 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.971649 8858 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.971731 8858 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:26.971856 8845 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.971932 8845 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 5977665e9a4d4174b1d8c7fba25f32a1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.972232 8846 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.972312 8846 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P 3513473af41a4ded9ea73b1a502fc778: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 04:03:26.973249 8893 tablet_replica.cc:439] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } } }
I20260926 04:03:26.973344 8893 tablet_replica.cc:442] T e73145da9f004f7c80b21914566fe55a P f51bbb4c6c27417a813a7dfbf5e761b1: This TxnStatusTablet replica's current role is: LEADER
I20260926 04:03:26.982915 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.979660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.0.200.130:33049)
0926 04:03:26.979799 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.979801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.979812 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:26.979889 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.979890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.979916 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.980366 (+ 450us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:26.980494 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.981253 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.982021 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.982289 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.982369 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.982509 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.982676 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.982756 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.982781 (+ 25us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.982805 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260926 04:03:26.988391 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.985807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.200.130:37119)
0926 04:03:26.986009 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.986021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.986097 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.986223 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.986224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.986225 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:26.986285 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.986286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.986960 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.986961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.987570 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.987572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.987960 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.987961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.988022 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.988034 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.988217 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.988219 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:03:26.988260 (+ 41us) client_negotiation.cc:240] Negotiation successful
0926 04:03:26.988307 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260926 04:03:26.988366 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.985883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:37119 (local address 127.0.200.131:35707)
0926 04:03:26.985956 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.985958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.986087 (+ 129us) server_negotiation.cc:407] Connection header received
0926 04:03:26.986153 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.986154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.986165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.986220 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:26.986348 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.986874 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.987656 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.987855 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.987941 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:26.988118 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:26.988157 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:26.988216 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:26.988261 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 04:03:26.988281 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 04:03:26.990736 8766 raft_consensus.cc:1261] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Refusing update from remote peer 5977665e9a4d4174b1d8c7fba25f32a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.991044 8486 raft_consensus.cc:1261] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Refusing update from remote peer 5977665e9a4d4174b1d8c7fba25f32a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:26.991144 8871 consensus_queue.cc:1035] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.991241 8845 consensus_queue.cc:1035] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:26.993254 8887 mvcc.cc:205] Tried to move back new op lower bound from 7333459587031605248 to 7333459585488732160. Current Snapshot: MvccSnapshot[applied={T|T < 7333459587031605248}]
I20260926 04:03:27.000624 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.997874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:46935 (local address 127.0.200.130:33049)
0926 04:03:26.997945 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:26.997947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:26.998044 (+ 97us) server_negotiation.cc:407] Connection header received
0926 04:03:26.998123 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:26.998124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:26.998136 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:26.998191 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:26.998342 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.998941 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.999715 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.999998 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.000084 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.000286 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.000357 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.000391 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.000524 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.000550 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 04:03:27.000658 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:26.997841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:33049 (local address 127.0.200.131:46935)
0926 04:03:26.997985 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:26.997996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:26.998057 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:26.998193 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:26.998194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:26.998196 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:26.998253 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.998255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:26.999032 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:26.999033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:26.999629 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:26.999631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.000084 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.000085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.000155 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.000172 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.000439 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.000441 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:03:27.000515 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 04:03:27.000566 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":88,"threads_started":1}
I20260926 04:03:27.011559 8327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.011682 8327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.012349 8327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Sending DeleteTablet for 3 replicas of tablet 7ec75bfba08643698d3d2dc7dc18d66d
I20260926 04:03:27.012661 8466 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260926 04:03:27.013008 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:60468
I20260926 04:03:27.013224 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec75bfba08643698d3d2dc7dc18d66d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35450
I20260926 04:03:27.013434 8910 tablet_replica.cc:330] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: stopping tablet replica
I20260926 04:03:27.013495 8910 raft_consensus.cc:2229] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.013537 8910 raft_consensus.cc:2258] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.013645 8911 tablet_replica.cc:330] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: stopping tablet replica
I20260926 04:03:27.013693 8911 raft_consensus.cc:2229] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:27.013775 8911 raft_consensus.cc:2258] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.013877 8912 tablet_replica.cc:330] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: stopping tablet replica
I20260926 04:03:27.013926 8912 raft_consensus.cc:2229] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.013959 8912 raft_consensus.cc:2258] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.014231 8912 ts_tablet_manager.cc:1905] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.016202 8912 ts_tablet_manager.cc:1918] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:27.016215 8910 ts_tablet_manager.cc:1905] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.016340 8912 log.cc:1196] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/7ec75bfba08643698d3d2dc7dc18d66d
I20260926 04:03:27.016671 8912 ts_tablet_manager.cc:1939] T 7ec75bfba08643698d3d2dc7dc18d66d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting consensus metadata
I20260926 04:03:27.017068 8316 catalog_manager.cc:4841] TS f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707): tablet 7ec75bfba08643698d3d2dc7dc18d66d (table test-table [id=db3859303fba42ae92040bc2a1985735]) successfully deleted
I20260926 04:03:27.017256 8911 ts_tablet_manager.cc:1905] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.020021 8911 ts_tablet_manager.cc:1918] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:27.020093 8911 log.cc:1196] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/7ec75bfba08643698d3d2dc7dc18d66d
I20260926 04:03:27.020323 8911 ts_tablet_manager.cc:1939] T 7ec75bfba08643698d3d2dc7dc18d66d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting consensus metadata
I20260926 04:03:27.020792 8313 catalog_manager.cc:4841] TS 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049): tablet 7ec75bfba08643698d3d2dc7dc18d66d (table test-table [id=db3859303fba42ae92040bc2a1985735]) successfully deleted
I20260926 04:03:27.020965 8910 ts_tablet_manager.cc:1918] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:27.021032 8910 log.cc:1196] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/7ec75bfba08643698d3d2dc7dc18d66d
I20260926 04:03:27.021255 8910 ts_tablet_manager.cc:1939] T 7ec75bfba08643698d3d2dc7dc18d66d P 3513473af41a4ded9ea73b1a502fc778: Deleting consensus metadata
I20260926 04:03:27.021788 8314 catalog_manager.cc:4841] TS 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749): tablet 7ec75bfba08643698d3d2dc7dc18d66d (table test-table [id=db3859303fba42ae92040bc2a1985735]) successfully deleted
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 04:03:27.047513 8380 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:27.040596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.0.200.190:45967)
0926 04:03:27.040645 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.040648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.040736 (+ 88us) server_negotiation.cc:407] Connection header received
0926 04:03:27.040807 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.040808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.040819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.040899 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:27.041633 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.042449 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.043534 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.043833 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.043907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.047441 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39914: BlockingRecv error: recv got EOF from 127.0.0.1:39914 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM
I20260926 04:03:27.057825 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.048765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.0.200.190:45967)
0926 04:03:27.048805 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.048807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.048817 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:27.048896 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.048897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.048906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.048970 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:27.049603 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.050330 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.051415 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.051691 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.051790 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.054775 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:27.054777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:27.054777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:27.054784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:27.056519 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:27.056936 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:27.056937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:27.056938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:27.056996 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:27.057201 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:27.057202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:27.057202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:27.057335 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:27.057419 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.057643 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.057725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:03:27.060292 8327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:27.060482 8327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:27.061908 8606 tablet_service.cc:1467] Processing CreateTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d (DEFAULT_TABLE table=test-table [id=f3e726883e4b4bf99c1e215b548c3101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.061916 8466 tablet_service.cc:1467] Processing CreateTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d (DEFAULT_TABLE table=test-table [id=f3e726883e4b4bf99c1e215b548c3101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.062043 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6acd1bb7986425b9167dc67dd6eb91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.062060 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6acd1bb7986425b9167dc67dd6eb91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.062803 8746 tablet_service.cc:1467] Processing CreateTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d (DEFAULT_TABLE table=test-table [id=f3e726883e4b4bf99c1e215b548c3101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.062917 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6acd1bb7986425b9167dc67dd6eb91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.064193 8844 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Bootstrap starting.
I20260926 04:03:27.064195 8841 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Bootstrap starting.
I20260926 04:03:27.064551 8844 tablet_bootstrap.cc:654] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.064634 8841 tablet_bootstrap.cc:654] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.065270 8841 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: No bootstrap required, opened a new log
I20260926 04:03:27.065315 8841 ts_tablet_manager.cc:1397] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:27.065475 8841 raft_consensus.cc:348] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.065541 8841 raft_consensus.cc:374] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.065560 8841 raft_consensus.cc:729] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.065601 8841 consensus_queue.cc:260] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.065685 8841 ts_tablet_manager.cc:1428] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.066279 8840 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Bootstrap starting.
I20260926 04:03:27.066948 8840 tablet_bootstrap.cc:654] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.067415 8844 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: No bootstrap required, opened a new log
I20260926 04:03:27.067461 8844 ts_tablet_manager.cc:1397] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260926 04:03:27.067593 8844 raft_consensus.cc:348] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.067657 8844 raft_consensus.cc:374] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.067675 8844 raft_consensus.cc:729] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.067734 8844 consensus_queue.cc:260] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.067819 8844 ts_tablet_manager.cc:1428] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.068238 8840 tablet_bootstrap.cc:492] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: No bootstrap required, opened a new log
I20260926 04:03:27.068292 8840 ts_tablet_manager.cc:1397] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:27.068496 8840 raft_consensus.cc:348] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.068567 8840 raft_consensus.cc:374] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.068584 8840 raft_consensus.cc:729] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.068625 8840 consensus_queue.cc:260] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.068696 8840 ts_tablet_manager.cc:1428] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.093037 8858 raft_consensus.cc:482] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:27.093173 8858 raft_consensus.cc:504] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.093389 8858 leader_election.cc:288] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049)
I20260926 04:03:27.093583 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6acd1bb7986425b9167dc67dd6eb91d" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778" is_pre_election: true
I20260926 04:03:27.093640 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6acd1bb7986425b9167dc67dd6eb91d" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" is_pre_election: true
I20260926 04:03:27.093691 8486 raft_consensus.cc:2454] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:27.093722 8626 raft_consensus.cc:2454] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 0.
I20260926 04:03:27.093873 8698 leader_election.cc:302] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3513473af41a4ded9ea73b1a502fc778, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:27.093986 8858 raft_consensus.cc:2781] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:27.094033 8858 raft_consensus.cc:482] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:27.094048 8858 raft_consensus.cc:3037] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.095011 8858 raft_consensus.cc:504] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.095156 8858 leader_election.cc:288] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Requested vote from peers 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749), 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049)
I20260926 04:03:27.095300 8486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6acd1bb7986425b9167dc67dd6eb91d" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3513473af41a4ded9ea73b1a502fc778"
I20260926 04:03:27.095391 8486 raft_consensus.cc:3037] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.095362 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6acd1bb7986425b9167dc67dd6eb91d" candidate_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
I20260926 04:03:27.095430 8626 raft_consensus.cc:3037] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.096202 8626 raft_consensus.cc:2454] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:27.096202 8486 raft_consensus.cc:2454] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51bbb4c6c27417a813a7dfbf5e761b1 in term 1.
I20260926 04:03:27.096390 8698 leader_election.cc:302] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3513473af41a4ded9ea73b1a502fc778, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:27.096498 8858 raft_consensus.cc:2781] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:27.096550 8858 raft_consensus.cc:686] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 LEADER]: Becoming Leader. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Running, Role: LEADER
I20260926 04:03:27.096599 8858 consensus_queue.cc:237] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.097299 8327 catalog_manager.cc:5495] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:27.106624 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.104354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.0.200.131:35707)
0926 04:03:27.104405 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.104408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.104417 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:27.104464 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.104465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.104476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.104530 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:27.104675 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.105150 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.105913 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.106108 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.106184 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.106341 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.106377 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.106435 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.106496 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.106514 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:03:27.107506 8626 raft_consensus.cc:1261] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:27.107546 8486 raft_consensus.cc:1261] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Refusing update from remote peer f51bbb4c6c27417a813a7dfbf5e761b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:27.107745 8896 consensus_queue.cc:1035] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:27.107892 8858 consensus_queue.cc:1035] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:27.110616 8328 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.110725 8328 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.111377 8328 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Sending DeleteTablet for 3 replicas of tablet e6acd1bb7986425b9167dc67dd6eb91d
I20260926 04:03:27.111688 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35450
I20260926 04:03:27.111685 8466 tablet_service.cc:1514] Processing DeleteTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260926 04:03:27.111785 8912 tablet_replica.cc:330] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: stopping tablet replica
I20260926 04:03:27.111842 8912 raft_consensus.cc:2229] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:27.111900 8912 raft_consensus.cc:2258] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.112074 8910 tablet_replica.cc:330] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: stopping tablet replica
I20260926 04:03:27.112144 8910 raft_consensus.cc:2229] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.112197 8910 raft_consensus.cc:2258] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.112347 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet e6acd1bb7986425b9167dc67dd6eb91d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:60468
I20260926 04:03:27.112368 8328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:27.112533 8328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:27.112723 8911 tablet_replica.cc:330] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: stopping tablet replica
I20260926 04:03:27.112788 8911 raft_consensus.cc:2229] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.112831 8911 raft_consensus.cc:2258] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.113334 8912 ts_tablet_manager.cc:1905] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.113598 8911 ts_tablet_manager.cc:1905] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.113816 8466 tablet_service.cc:1467] Processing CreateTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff (DEFAULT_TABLE table=test-table [id=d2345a4a0cfd41d2aa59f0e75bff7c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.113941 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.115077 8912 ts_tablet_manager.cc:1918] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:27.115134 8912 log.cc:1196] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/e6acd1bb7986425b9167dc67dd6eb91d
I20260926 04:03:27.115154 8911 ts_tablet_manager.cc:1918] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:27.115206 8911 log.cc:1196] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/e6acd1bb7986425b9167dc67dd6eb91d
I20260926 04:03:27.115357 8912 ts_tablet_manager.cc:1939] T e6acd1bb7986425b9167dc67dd6eb91d P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting consensus metadata
I20260926 04:03:27.115391 8840 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: Bootstrap starting.
I20260926 04:03:27.115440 8911 ts_tablet_manager.cc:1939] T e6acd1bb7986425b9167dc67dd6eb91d P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting consensus metadata
I20260926 04:03:27.116058 8840 tablet_bootstrap.cc:654] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.116590 8313 catalog_manager.cc:4841] TS 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049): tablet e6acd1bb7986425b9167dc67dd6eb91d (table test-table [id=f3e726883e4b4bf99c1e215b548c3101]) successfully deleted
I20260926 04:03:27.116676 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff (DEFAULT_TABLE table=test-table [id=d2345a4a0cfd41d2aa59f0e75bff7c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.116777 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.117806 8841 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Bootstrap starting.
I20260926 04:03:27.118201 8841 tablet_bootstrap.cc:654] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.118952 8840 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: No bootstrap required, opened a new log
I20260926 04:03:27.118996 8840 ts_tablet_manager.cc:1397] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 04:03:27.119165 8840 raft_consensus.cc:348] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.119230 8840 raft_consensus.cc:374] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.119243 8840 raft_consensus.cc:729] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.119297 8840 consensus_queue.cc:260] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.119387 8840 ts_tablet_manager.cc:1428] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.119575 8841 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: No bootstrap required, opened a new log
I20260926 04:03:27.119625 8841 ts_tablet_manager.cc:1397] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:27.119736 8841 raft_consensus.cc:348] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.119781 8841 raft_consensus.cc:374] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.119793 8841 raft_consensus.cc:729] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5977665e9a4d4174b1d8c7fba25f32a1, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.119853 8841 consensus_queue.cc:260] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.119936 8841 ts_tablet_manager.cc:1428] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.120170 8316 catalog_manager.cc:4841] TS f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707): tablet e6acd1bb7986425b9167dc67dd6eb91d (table test-table [id=f3e726883e4b4bf99c1e215b548c3101]) successfully deleted
I20260926 04:03:27.120399 8746 tablet_service.cc:1467] Processing CreateTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff (DEFAULT_TABLE table=test-table [id=d2345a4a0cfd41d2aa59f0e75bff7c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:27.120519 8746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:27.120762 8910 ts_tablet_manager.cc:1905] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.121544 8844 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Bootstrap starting.
I20260926 04:03:27.121887 8844 tablet_bootstrap.cc:654] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:27.122423 8910 ts_tablet_manager.cc:1918] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:27.122521 8910 log.cc:1196] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/e6acd1bb7986425b9167dc67dd6eb91d
I20260926 04:03:27.122542 8844 tablet_bootstrap.cc:492] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: No bootstrap required, opened a new log
I20260926 04:03:27.122610 8844 ts_tablet_manager.cc:1397] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:27.122723 8844 raft_consensus.cc:348] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.122808 8844 raft_consensus.cc:374] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:27.122816 8910 ts_tablet_manager.cc:1939] T e6acd1bb7986425b9167dc67dd6eb91d P 3513473af41a4ded9ea73b1a502fc778: Deleting consensus metadata
I20260926 04:03:27.122843 8844 raft_consensus.cc:729] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51bbb4c6c27417a813a7dfbf5e761b1, State: Initialized, Role: FOLLOWER
I20260926 04:03:27.122888 8844 consensus_queue.cc:260] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.122982 8844 ts_tablet_manager.cc:1428] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:27.123157 8314 catalog_manager.cc:4841] TS 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749): tablet e6acd1bb7986425b9167dc67dd6eb91d (table test-table [id=f3e726883e4b4bf99c1e215b548c3101]) successfully deleted
I20260926 04:03:27.320178 8846 raft_consensus.cc:482] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:27.320299 8846 raft_consensus.cc:504] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.320664 8846 leader_election.cc:288] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049), f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707)
I20260926 04:03:27.323943 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.320971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:51659 (local address 127.0.200.131:35707)
0926 04:03:27.321040 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.321043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.321154 (+ 111us) server_negotiation.cc:407] Connection header received
0926 04:03:27.321216 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.321217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.321229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.321300 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:27.321462 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.321942 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323195 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.323571 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323669 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.323682 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.323828 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.323865 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 04:03:27.323995 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.321058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:59837 (local address 127.0.200.130:33049)
0926 04:03:27.321118 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.321121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.321179 (+ 58us) server_negotiation.cc:407] Connection header received
0926 04:03:27.321256 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.321257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.321265 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.321337 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:27.321504 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.321976 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323213 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.323598 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323685 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.323693 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.323851 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.323892 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 04:03:27.324039 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.320899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.200.129:51659)
0926 04:03:27.321058 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:27.321070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:27.321148 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:27.321304 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:27.321305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:27.321307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:27.321388 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.321390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.322081 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.322082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.323096 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.323099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.323669 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.323750 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.323754 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:27.323822 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:03:27.323865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":101,"thread_start_us":78,"threads_started":1}
I20260926 04:03:27.324348 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b1a3e5ef714edaa8001c3c2c84a2ff" candidate_uuid: "3513473af41a4ded9ea73b1a502fc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" is_pre_election: true
I20260926 04:03:27.324465 8766 raft_consensus.cc:2454] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3513473af41a4ded9ea73b1a502fc778 in term 0.
I20260926 04:03:27.324524 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.320986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:33049 (local address 127.0.200.129:59837)
0926 04:03:27.321089 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:27.321101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:27.321176 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:27.321339 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:27.321340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:27.321342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:27.321433 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.321435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.322081 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.322082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.323095 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.323098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.323696 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.323698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.323763 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.323766 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:27.323841 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 04:03:27.323886 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":46,"thread_start_us":58,"threads_started":1}
I20260926 04:03:27.324745 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b1a3e5ef714edaa8001c3c2c84a2ff" candidate_uuid: "3513473af41a4ded9ea73b1a502fc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" is_pre_election: true
I20260926 04:03:27.324774 8420 leader_election.cc:302] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3513473af41a4ded9ea73b1a502fc778, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:27.324831 8626 raft_consensus.cc:2454] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3513473af41a4ded9ea73b1a502fc778 in term 0.
I20260926 04:03:27.324915 8846 raft_consensus.cc:2781] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:27.324958 8846 raft_consensus.cc:482] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:27.324985 8846 raft_consensus.cc:3037] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.325677 8846 raft_consensus.cc:504] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.325852 8846 leader_election.cc:288] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [CANDIDATE]: Term 1 election: Requested vote from peers 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049), f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707)
I20260926 04:03:27.325992 8766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b1a3e5ef714edaa8001c3c2c84a2ff" candidate_uuid: "3513473af41a4ded9ea73b1a502fc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1"
I20260926 04:03:27.326035 8626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b1a3e5ef714edaa8001c3c2c84a2ff" candidate_uuid: "3513473af41a4ded9ea73b1a502fc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5977665e9a4d4174b1d8c7fba25f32a1"
I20260926 04:03:27.326092 8766 raft_consensus.cc:3037] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.326120 8626 raft_consensus.cc:3037] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:27.326839 8766 raft_consensus.cc:2454] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3513473af41a4ded9ea73b1a502fc778 in term 1.
I20260926 04:03:27.327180 8420 leader_election.cc:302] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3513473af41a4ded9ea73b1a502fc778, f51bbb4c6c27417a813a7dfbf5e761b1; no voters:
I20260926 04:03:27.327255 8626 raft_consensus.cc:2454] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3513473af41a4ded9ea73b1a502fc778 in term 1.
I20260926 04:03:27.327395 8846 raft_consensus.cc:2781] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:27.327621 8846 raft_consensus.cc:686] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 1 LEADER]: Becoming Leader. State: Replica: 3513473af41a4ded9ea73b1a502fc778, State: Running, Role: LEADER
I20260926 04:03:27.327869 8846 consensus_queue.cc:237] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } }
I20260926 04:03:27.329003 8327 catalog_manager.cc:5495] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129). New cstate: current_term: 1 leader_uuid: "3513473af41a4ded9ea73b1a502fc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3513473af41a4ded9ea73b1a502fc778" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 } health_report { overall_health: UNKNOWN } } }
I20260926 04:03:27.356446 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.353607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.0.200.129:44749)
0926 04:03:27.353827 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.353829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.353837 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:27.353906 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.353907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.353928 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.354009 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:27.354170 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.354716 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.355494 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.355732 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.355791 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.355961 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.356091 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.356150 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.356243 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.356260 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260926 04:03:27.362414 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.360009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:52089 (local address 127.0.200.131:35707)
0926 04:03:27.360093 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.360096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.360106 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:27.360130 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.360131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.360141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.360215 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:27.360377 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.360867 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.361693 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.361899 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.361977 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.362146 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.362186 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.362247 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.362310 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.362331 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 04:03:27.362455 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.359755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35707 (local address 127.0.200.129:52089)
0926 04:03:27.359983 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:27.359993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:27.360061 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:27.360217 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:27.360218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:27.360219 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:27.360297 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.360299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.361000 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.361001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.361597 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.361599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.361976 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.361977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.362045 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.362059 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.362249 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.362252 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:27.362294 (+ 42us) client_negotiation.cc:240] Negotiation successful
0926 04:03:27.362342 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":157,"threads_started":1}
I20260926 04:03:27.364814 8766 raft_consensus.cc:1261] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Refusing update from remote peer 3513473af41a4ded9ea73b1a502fc778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:27.364873 8626 raft_consensus.cc:1261] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Refusing update from remote peer 3513473af41a4ded9ea73b1a502fc778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:27.365152 8937 consensus_queue.cc:1035] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51bbb4c6c27417a813a7dfbf5e761b1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:27.365283 8846 consensus_queue.cc:1035] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5977665e9a4d4174b1d8c7fba25f32a1" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:27.372730 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.370229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44749 (local address 127.0.200.131:34671)
0926 04:03:27.370277 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:27.370290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:27.370331 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:27.370530 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:27.370530 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:27.370532 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 04:03:27.370607 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.370609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.371241 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.371242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.371788 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:27.371789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.372249 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.372250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:27.372325 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.372333 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.372562 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.372564 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:03:27.372614 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 04:03:27.372664 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 04:03:27.372722 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.370337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:34671 (local address 127.0.200.129:44749)
0926 04:03:27.370433 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:27.370436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:27.370444 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:27.370466 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:27.370467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:27.370477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:27.370529 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:27.370659 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.371146 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.371883 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:27.372159 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:27.372249 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:27.372451 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:27.372492 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:27.372546 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:27.372621 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 04:03:27.372645 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 04:03:27.379550 8327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.379680 8327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:27.380352 8327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 330ab62981a447eda5db212f4b282e27: Sending DeleteTablet for 3 replicas of tablet 73b1a3e5ef714edaa8001c3c2c84a2ff
I20260926 04:03:27.380615 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:60468
I20260926 04:03:27.380705 8746 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35450
I20260926 04:03:27.380774 8466 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:27 UTC) from {username='oryx', principal='oryx/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260926 04:03:27.380801 8911 tablet_replica.cc:330] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: stopping tablet replica
I20260926 04:03:27.380862 8911 raft_consensus.cc:2229] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.380909 8911 raft_consensus.cc:2258] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.381004 8912 tablet_replica.cc:330] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: stopping tablet replica
I20260926 04:03:27.381063 8912 raft_consensus.cc:2229] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:27.381108 8912 raft_consensus.cc:2258] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.381178 8910 tablet_replica.cc:330] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778: stopping tablet replica
I20260926 04:03:27.381232 8910 raft_consensus.cc:2229] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:27.381321 8910 raft_consensus.cc:2258] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 3513473af41a4ded9ea73b1a502fc778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:27.381502 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8401
I20260926 04:03:27.383119 8912 ts_tablet_manager.cc:1905] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.385015 8911 ts_tablet_manager.cc:1905] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:27.385020 8912 ts_tablet_manager.cc:1918] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:27.385082 8912 log.cc:1196] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/73b1a3e5ef714edaa8001c3c2c84a2ff
I20260926 04:03:27.385322 8912 ts_tablet_manager.cc:1939] T 73b1a3e5ef714edaa8001c3c2c84a2ff P f51bbb4c6c27417a813a7dfbf5e761b1: Deleting consensus metadata
I20260926 04:03:27.385895 8316 catalog_manager.cc:4841] TS f51bbb4c6c27417a813a7dfbf5e761b1 (127.0.200.131:35707): tablet 73b1a3e5ef714edaa8001c3c2c84a2ff (table test-table [id=d2345a4a0cfd41d2aa59f0e75bff7c04]) successfully deleted
I20260926 04:03:27.386380 8911 ts_tablet_manager.cc:1918] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 04:03:27.386431 8911 log.cc:1196] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/73b1a3e5ef714edaa8001c3c2c84a2ff
I20260926 04:03:27.386710 8911 ts_tablet_manager.cc:1939] T 73b1a3e5ef714edaa8001c3c2c84a2ff P 5977665e9a4d4174b1d8c7fba25f32a1: Deleting consensus metadata
I20260926 04:03:27.387197 8313 catalog_manager.cc:4841] TS 5977665e9a4d4174b1d8c7fba25f32a1 (127.0.200.130:33049): tablet 73b1a3e5ef714edaa8001c3c2c84a2ff (table test-table [id=d2345a4a0cfd41d2aa59f0e75bff7c04]) successfully deleted
W20260926 04:03:27.389432 8314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:44749 (error 108)
I20260926 04:03:27.389654 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8541
I20260926 04:03:27.390319 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:27.389830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44749 (local address 127.0.0.1:55836)
0926 04:03:27.390250 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:27.390286 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":81,"threads_started":1}
W20260926 04:03:27.390440 8314 catalog_manager.cc:4584] TS 3513473af41a4ded9ea73b1a502fc778 (127.0.200.129:44749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73b1a3e5ef714edaa8001c3c2c84a2ff: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44749: connect: Connection refused (error 111)
I20260926 04:03:27.396135 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8681
I20260926 04:03:27.402209 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8297
2026-09-26T04:03:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3055 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8951](info): set up 2 sockets
Sep 26 04:03:27 dist-test-slave-j8c1 krb5kdc[8951](info): commencing operation
krb5kdc: starting...
W20260926 04:03:28.463593 802 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:28 dist-test-slave-j8c1 krb5kdc[8951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:28.603372 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39457
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:28.691448 8964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:28.691736 8964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:28.691779 8964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:28.693094 8964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:28.693176 8964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:28.693200 8964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:28.693219 8964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:28.693246 8964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:28.694984 8964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:28.695303 8964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:28.695552 8964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:28.700976 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:28.700953 8972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:28.700953 8969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:28.701840 8964 server_base.cc:1034] running on GCE node
I20260926 04:03:28.702131 8964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:28.702455 8964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:28.703620 8964 hybrid_clock.cc:648] HybridClock initialized: now 1790395408703606 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T04:03:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1308 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:28 dist-test-slave-j8c1 krb5kdc[8977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:28 dist-test-slave-j8c1 krb5kdc[8977](info): set up 2 sockets
Sep 26 04:03:28 dist-test-slave-j8c1 krb5kdc[8977](info): commencing operation
krb5kdc: starting...
W20260926 04:03:29.824263 802 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:29 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:29.967864 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45793
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:30.060303 8990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:30.060540 8990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:30.060564 8990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:30.061817 8990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:30.061877 8990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:30.061900 8990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:30.061914 8990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:30.061928 8990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:30.063865 8990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:30.064186 8990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:30.064468 8990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:30.070312 8995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.070261 8998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.070261 8996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:30.071347 8990 server_base.cc:1034] running on GCE node
I20260926 04:03:30.071650 8990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:30.072001 8990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:30.073325 8990 hybrid_clock.cc:648] HybridClock initialized: now 1790395410073228 us; error 34 us; skew 500 ppm
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:30.084578 8990 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:30.085191 8990 webserver.cc:460] Webserver started at http://127.0.200.190:36451/ using document root <none> and password file <none>
I20260926 04:03:30.085485 8990 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:30.085541 8990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:30.085675 8990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:30.086935 8990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "a420cfc10db9454fb9f2a8484449ae9c"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "6ec9304addab2e1f5954f245490b4554"
server_key_iv: "aa51dcf3f927e59b1960ffbb31399a03"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.087332 8990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "a420cfc10db9454fb9f2a8484449ae9c"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "6ec9304addab2e1f5954f245490b4554"
server_key_iv: "aa51dcf3f927e59b1960ffbb31399a03"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.089067 8990 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 04:03:30.090517 9005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.090811 8990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:30.091161 8990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "a420cfc10db9454fb9f2a8484449ae9c"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "6ec9304addab2e1f5954f245490b4554"
server_key_iv: "aa51dcf3f927e59b1960ffbb31399a03"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.091285 8990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:30.130761 8990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:30.133339 8990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:30.133477 8990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:30.138321 8990 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:45793
I20260926 04:03:30.138407 9057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:45793 every 8 connection(s)
I20260926 04:03:30.138969 8990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:30.139858 9058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:30.142292 9058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Bootstrap starting.
I20260926 04:03:30.143006 9058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:30.143319 9058 log.cc:824] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:30.144395 9058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: No bootstrap required, opened a new log
I20260926 04:03:30.145567 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 8990
I20260926 04:03:30.145650 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:30.145870 802 external_mini_cluster.cc:1428] Setting key 44e31a60f7810435737ed86f63216f7e
I20260926 04:03:30.146466 9058 raft_consensus.cc:348] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:30.146695 9058 raft_consensus.cc:374] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:30.146739 9058 raft_consensus.cc:729] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a420cfc10db9454fb9f2a8484449ae9c, State: Initialized, Role: FOLLOWER
I20260926 04:03:30.146944 9058 consensus_queue.cc:260] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:30.147056 9058 raft_consensus.cc:388] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:30.147094 9058 raft_consensus.cc:482] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:30.147158 9058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:30.147784 9058 raft_consensus.cc:504] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:30.147919 9058 leader_election.cc:302] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a420cfc10db9454fb9f2a8484449ae9c; no voters:
I20260926 04:03:30.148140 9058 leader_election.cc:288] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:30.148175 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:30.148286 9063 raft_consensus.cc:686] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 LEADER]: Becoming Leader. State: Replica: a420cfc10db9454fb9f2a8484449ae9c, State: Running, Role: LEADER
I20260926 04:03:30.148377 9063 consensus_queue.cc:237] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:30.148432 9058 sys_catalog.cc:564] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:30.148835 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a420cfc10db9454fb9f2a8484449ae9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } } }
I20260926 04:03:30.148934 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:30.149065 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a420cfc10db9454fb9f2a8484449ae9c. Latest consensus state: current_term: 1 leader_uuid: "a420cfc10db9454fb9f2a8484449ae9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } } }
I20260926 04:03:30.149135 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: This master's current role is: LEADER
W20260926 04:03:30.150568 9079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:03:30.150640 9079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:03:30.150694 9068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:30.150913 9068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:30.152495 9068 catalog_manager.cc:1347] Generated new cluster ID: 9cbd2a1675e944beb468f3d8572223f7
I20260926 04:03:30.152567 9068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:30.161490 9068 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:30.161976 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.146659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.0.200.190:45793)
0926 04:03:30.147341 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:30.147352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:30.147391 (+ 39us) server_negotiation.cc:407] Connection header received
0926 04:03:30.147550 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:30.147557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:30.147615 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:30.147798 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:30.149216 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.150768 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.151912 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.152517 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.152633 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.156168 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:30.156175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:30.156175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:30.156192 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:30.159767 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.160333 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.160337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.160338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.160471 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.160901 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.160902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.160902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.161211 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:30.161434 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:30.161510 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 04:03:30.161761 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":569,"threads_started":1}
I20260926 04:03:30.162402 9068 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:30.167446 9068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Generated new TSK 0
I20260926 04:03:30.167683 9068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:30.203691 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:30.294242 9084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:30.294507 9084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:30.294603 9084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:30.296058 9084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:30.296128 9084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:30.296175 9084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:30.297829 9084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:30.298092 9084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:30.298374 9084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:30.305439 9090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:30.305605 9084 server_base.cc:1034] running on GCE node
W20260926 04:03:30.305447 9089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.305449 9092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:30.306033 9084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:30.306627 9084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:30.307897 9084 hybrid_clock.cc:648] HybridClock initialized: now 1790395410307886 us; error 36 us; skew 500 ppm
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:30.311725 9084 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:30.312333 9084 webserver.cc:460] Webserver started at http://127.0.200.129:33041/ using document root <none> and password file <none>
I20260926 04:03:30.312811 9084 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:30.312922 9084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:30.313107 9084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:30.314452 9084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "2b16453e840d4dad97b4d9b1a7f6d7d8"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "e6e84d9d6acf791331bdea5006cda105"
server_key_iv: "93b9ef17f83d413f72d81ad8afc8c962"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.315248 9084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "2b16453e840d4dad97b4d9b1a7f6d7d8"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "e6e84d9d6acf791331bdea5006cda105"
server_key_iv: "93b9ef17f83d413f72d81ad8afc8c962"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.317453 9084 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260926 04:03:30.318650 9099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.318856 9084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:30.318969 9084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "2b16453e840d4dad97b4d9b1a7f6d7d8"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "e6e84d9d6acf791331bdea5006cda105"
server_key_iv: "93b9ef17f83d413f72d81ad8afc8c962"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.319074 9084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:30.335589 9084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:30.338266 9084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:30.338591 9084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:30.338894 9084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:30.339246 9084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:30.339309 9084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.339354 9084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:30.339404 9084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.345871 9084 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:33011
I20260926 04:03:30.345916 9212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:33011 every 8 connection(s)
I20260926 04:03:30.346266 9084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:30.350446 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9084
I20260926 04:03:30.350607 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:30.350841 802 external_mini_cluster.cc:1428] Setting key ccc267b740e553391b97c07a2ce78b2f
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:30.360917 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.347281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44793 (local address 127.0.200.190:45793)
0926 04:03:30.347432 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:30.347434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:30.347659 (+ 225us) server_negotiation.cc:407] Connection header received
0926 04:03:30.348947 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:30.348949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:30.348963 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:30.349102 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:30.350085 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.350953 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.352210 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.352698 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.352794 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.356464 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:30.356466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:30.356467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:30.356481 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:30.359021 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.359598 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.359600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.359600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.359683 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.360033 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.360034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.360034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.360199 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:30.360281 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:30.360683 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 04:03:30.360797 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 04:03:30.361140 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.347227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:44793)
0926 04:03:30.347477 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:30.347534 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:30.348824 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:30.349194 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:30.349199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:30.349646 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:30.349961 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.349967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.351191 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.351193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.352098 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.352101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.352833 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.352834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.352957 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.353717 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:30.353733 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:30.356310 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:30.359168 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.359180 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.359183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.359498 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.359807 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.359808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.359808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.359949 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.360352 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:30.360359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:30.360513 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 04:03:30.360630 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 04:03:30.360802 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":155,"threads_started":1}
I20260926 04:03:30.361840 9213 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45793
I20260926 04:03:30.361963 9213 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:30.362178 9213 heartbeater.cc:511] Master 127.0.200.190:45793 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:03:30.363184 9022 ts_manager.cc:194] Registered new tserver with Master: 2b16453e840d4dad97b4d9b1a7f6d7d8 (127.0.200.129:33011)
I20260926 04:03:30.363929 9022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:44793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:30.387135 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:30.479837 9218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:30.480077 9218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:30.480103 9218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:30.481693 9218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:30.481758 9218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:30.481789 9218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:30.483549 9218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:30.483834 9218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:30.484103 9218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:30.489913 9223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.489928 9226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.489930 9224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:30.489980 9218 server_base.cc:1034] running on GCE node
I20260926 04:03:30.490820 9218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:30.491122 9218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:30.492309 9218 hybrid_clock.cc:648] HybridClock initialized: now 1790395410492298 us; error 37 us; skew 500 ppm
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:30.495371 9218 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:30.495942 9218 webserver.cc:460] Webserver started at http://127.0.200.130:42845/ using document root <none> and password file <none>
I20260926 04:03:30.496197 9218 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:30.496263 9218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:30.496448 9218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:30.497717 9218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "cbd34e6c3e2044e5b7f27ae39e2b1aee"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "673a990d8a33edc2096987737029aa1a"
server_key_iv: "a26db65b300000db00b68785f875fd36"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.498041 9218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "cbd34e6c3e2044e5b7f27ae39e2b1aee"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "673a990d8a33edc2096987737029aa1a"
server_key_iv: "a26db65b300000db00b68785f875fd36"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.499626 9218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260926 04:03:30.500676 9233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.500947 9218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:30.501053 9218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "cbd34e6c3e2044e5b7f27ae39e2b1aee"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "673a990d8a33edc2096987737029aa1a"
server_key_iv: "a26db65b300000db00b68785f875fd36"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.501122 9218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:30.536948 9218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:30.540367 9218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:30.540530 9218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:30.540834 9218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:30.541253 9218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:30.541293 9218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.541322 9218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:30.541335 9218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.549474 9218 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:46251
I20260926 04:03:30.549587 9346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:46251 every 8 connection(s)
I20260926 04:03:30.549961 9218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:30.554363 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9218
I20260926 04:03:30.554530 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:30.554737 802 external_mini_cluster.cc:1428] Setting key 4d10b327a019c7e82343ad595a038030
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:30.565934 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.551245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:34499 (local address 127.0.200.190:45793)
0926 04:03:30.551312 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:30.551315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:30.551583 (+ 268us) server_negotiation.cc:407] Connection header received
0926 04:03:30.553063 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:30.553065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:30.553079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:30.553167 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:30.554440 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.555206 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.556765 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.557050 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.557150 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.561029 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:30.561031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:30.561031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:30.561040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:30.563639 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.564411 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.564413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.564413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.564499 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.564943 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.564944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.564944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.565114 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:30.565199 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:30.565709 (+ 510us) server_negotiation.cc:299] Negotiation successful
0926 04:03:30.565813 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 04:03:30.566078 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.551120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:34499)
0926 04:03:30.551345 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:30.551398 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:30.552834 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:30.553330 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:30.553336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:30.553970 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:30.554267 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.554276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.555318 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.555319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.556610 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.556614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.557210 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.557211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.557438 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.558808 (+ 1370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:30.558820 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:30.560851 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:30.563823 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.563839 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.563843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.564279 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.564649 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.564651 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.564652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.564848 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.565234 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:30.565241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:30.565527 (+ 286us) client_negotiation.cc:769] Sending connection context
0926 04:03:30.565651 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 04:03:30.565829 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":139,"threads_started":1}
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:03:30.567157 9347 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45793
I20260926 04:03:30.567317 9347 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:30.567564 9347 heartbeater.cc:511] Master 127.0.200.190:45793 requested a full tablet report, sending...
I20260926 04:03:30.568185 9022 ts_manager.cc:194] Registered new tserver with Master: cbd34e6c3e2044e5b7f27ae39e2b1aee (127.0.200.130:46251)
I20260926 04:03:30.568594 9022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:34499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:30.591123 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:30.688293 9352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:30.688525 9352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:30.688551 9352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:30.689807 9352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:30.689867 9352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:30.689896 9352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:30.691546 9352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:30.691840 9352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:30.692108 9352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:30.698017 9358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.698081 9360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:30.697968 9357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:30.698477 9352 server_base.cc:1034] running on GCE node
I20260926 04:03:30.698773 9352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:30.699100 9352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:30.700284 9352 hybrid_clock.cc:648] HybridClock initialized: now 1790395410700240 us; error 62 us; skew 500 ppm
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:30.703827 9352 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:30.704551 9352 webserver.cc:460] Webserver started at http://127.0.200.131:38543/ using document root <none> and password file <none>
I20260926 04:03:30.704821 9352 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:30.704896 9352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:30.705050 9352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:30.706283 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "8929dbe783ca41a79fd09678becf7f32"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "15839aaa6072e153df25c8b71772b1f5"
server_key_iv: "42061b551b8a9be5381c8598460d2bd1"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.706756 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "8929dbe783ca41a79fd09678becf7f32"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "15839aaa6072e153df25c8b71772b1f5"
server_key_iv: "42061b551b8a9be5381c8598460d2bd1"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.708362 9352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:30.709434 9367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.709681 9352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:30.709805 9352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "8929dbe783ca41a79fd09678becf7f32"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "15839aaa6072e153df25c8b71772b1f5"
server_key_iv: "42061b551b8a9be5381c8598460d2bd1"
server_key_version: "encryptionkey@0"
I20260926 04:03:30.709904 9352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:30.742662 9352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:30.746088 9352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:30.746364 9352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:30.746690 9352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:30.747044 9352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:30.747102 9352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.747149 9352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:30.747175 9352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:30.755568 9352 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:44273
I20260926 04:03:30.755802 9480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:44273 every 8 connection(s)
I20260926 04:03:30.756102 9352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:30.757783 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9352
I20260926 04:03:30.757903 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:30.758245 802 external_mini_cluster.cc:1428] Setting key 3fa9b0804a58cb79f50fe29d3d589bdf
Sep 26 04:03:30 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:30.770462 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.757390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:56901 (local address 127.0.200.190:45793)
0926 04:03:30.757457 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:30.757460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:30.757677 (+ 217us) server_negotiation.cc:407] Connection header received
0926 04:03:30.759084 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:30.759086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:30.759102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:30.759170 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:30.760239 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.760785 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.762423 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.762773 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.762854 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.766142 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:30.766144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:30.766145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:30.766154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:30.768312 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.769024 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.769026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.769026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.769110 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:30.769525 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:30.769526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:30.769526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:30.769679 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:30.769753 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:30.770245 (+ 492us) server_negotiation.cc:299] Negotiation successful
0926 04:03:30.770356 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 04:03:30.770818 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:30.757290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:56901)
0926 04:03:30.757535 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:30.757587 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:30.758941 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:30.759284 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:30.759289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:30.759783 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:30.760117 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.760124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.760889 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.760889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.762137 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:30.762139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:30.762879 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:30.762881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:30.763017 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:30.763720 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:30.763729 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:30.765991 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:30.768518 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.768528 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.768531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.768892 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.769238 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:30.769239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:30.769240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:30.769429 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:30.769851 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:30.769858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:30.770018 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 04:03:30.770144 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 04:03:30.770367 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":145,"threads_started":1}
I20260926 04:03:30.771277 9481 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45793
I20260926 04:03:30.771390 9481 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:30.771582 9481 heartbeater.cc:511] Master 127.0.200.190:45793 requested a full tablet report, sending...
I20260926 04:03:30.772022 9022 ts_manager.cc:194] Registered new tserver with Master: 8929dbe783ca41a79fd09678becf7f32 (127.0.200.131:44273)
I20260926 04:03:30.772464 9022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:56901
I20260926 04:03:30.782903 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
Entry for principal oryx/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab.
WARNING: no policy specified for oryx/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
Entry for principal oryx/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab.
WARNING: no policy specified for oryx/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
Entry for principal oryx/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab.
I20260926 04:03:30.880106 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9084
I20260926 04:03:30.885272 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9218
I20260926 04:03:30.890715 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9352
I20260926 04:03:30.895663 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 8990
I20260926 04:03:30.901664 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45793
--webserver_interface=127.0.200.190
--webserver_port=36451
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:30.990772 9490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:30.991124 9490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:30.991170 9490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:30.993868 9490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:30.993995 9490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:30.994024 9490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:30.994050 9490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:30.994081 9490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:30.996433 9490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=36451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:30.996840 9490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:30.997119 9490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:31.002908 9496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.002974 9490 server_base.cc:1034] running on GCE node
W20260926 04:03:31.002991 9495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:31.002908 9498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.003414 9490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:31.003750 9490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:31.004971 9490 hybrid_clock.cc:648] HybridClock initialized: now 1790395411004918 us; error 75 us; skew 500 ppm
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395411, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:31.008790 9490 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:31.009382 9490 webserver.cc:460] Webserver started at http://127.0.200.190:36451/ using document root <none> and password file <none>
I20260926 04:03:31.009656 9490 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:31.009765 9490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:31.011547 9490 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:03:31.012477 9505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.012741 9490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:31.012863 9490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "a420cfc10db9454fb9f2a8484449ae9c"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "6ec9304addab2e1f5954f245490b4554"
server_key_iv: "aa51dcf3f927e59b1960ffbb31399a03"
server_key_version: "encryptionkey@0"
I20260926 04:03:31.013293 9490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:31.021025 9490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:31.024483 9490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:31.024710 9490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:31.030866 9490 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:45793
I20260926 04:03:31.030908 9557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:45793 every 8 connection(s)
I20260926 04:03:31.031507 9490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:31.033429 9558 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:03:31.034237 9558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Bootstrap starting.
I20260926 04:03:31.037955 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9490
I20260926 04:03:31.038311 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:33011
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:45793
--webserver_port=33041
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:03:31.043502 9558 log.cc:824] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:31.045517 9558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:03:31.045855 9558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Bootstrap complete.
I20260926 04:03:31.048918 9558 raft_consensus.cc:348] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:31.049259 9558 raft_consensus.cc:729] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a420cfc10db9454fb9f2a8484449ae9c, State: Initialized, Role: FOLLOWER
I20260926 04:03:31.049489 9558 consensus_queue.cc:260] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:31.049572 9558 raft_consensus.cc:388] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:31.049631 9558 raft_consensus.cc:482] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:31.049688 9558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:03:31.051203 9558 raft_consensus.cc:504] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:31.051338 9558 leader_election.cc:302] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a420cfc10db9454fb9f2a8484449ae9c; no voters:
I20260926 04:03:31.051594 9558 leader_election.cc:288] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 04:03:31.051649 9562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 2 FOLLOWER]: Leader election won for term 2
I20260926 04:03:31.051813 9562 raft_consensus.cc:686] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [term 2 LEADER]: Becoming Leader. State: Replica: a420cfc10db9454fb9f2a8484449ae9c, State: Running, Role: LEADER
I20260926 04:03:31.051956 9558 sys_catalog.cc:564] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:31.051963 9562 consensus_queue.cc:237] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } }
I20260926 04:03:31.052495 9563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a420cfc10db9454fb9f2a8484449ae9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } } }
I20260926 04:03:31.052593 9563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:31.052814 9564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a420cfc10db9454fb9f2a8484449ae9c. Latest consensus state: current_term: 2 leader_uuid: "a420cfc10db9454fb9f2a8484449ae9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a420cfc10db9454fb9f2a8484449ae9c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45793 } } }
I20260926 04:03:31.052912 9564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:31.053066 9568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:31.053818 9568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:31.054056 9568 catalog_manager.cc:1259] Loaded cluster ID: 9cbd2a1675e944beb468f3d8572223f7
I20260926 04:03:31.054332 9568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:31.055348 9568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:31.055563 9568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a420cfc10db9454fb9f2a8484449ae9c: Loaded TSK: 0
I20260926 04:03:31.056027 9568 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 04:03:31.144107 9560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:31.144330 9560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:31.144358 9560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:31.145689 9560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:31.145741 9560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:31.145772 9560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:31.147356 9560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.129.keytab
--principal=oryx/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:33011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=33041
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:31.147615 9560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:31.147883 9560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:31.153546 9587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.153772 9560 server_base.cc:1034] running on GCE node
W20260926 04:03:31.153566 9584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:31.154045 9585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.154475 9560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:31.154909 9560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:31.156147 9560 hybrid_clock.cc:648] HybridClock initialized: now 1790395411156109 us; error 53 us; skew 500 ppm
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395411, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:31.159876 9560 init.cc:377] Logged in from keytab as oryx/127.0.200.129@KRBTEST.COM (short username oryx)
I20260926 04:03:31.160573 9560 webserver.cc:460] Webserver started at http://127.0.200.129:33041/ using document root <none> and password file <none>
I20260926 04:03:31.160846 9560 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:31.160938 9560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:31.162415 9560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:31.163121 9594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.163403 9560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:03:31.163511 9560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "2b16453e840d4dad97b4d9b1a7f6d7d8"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "e6e84d9d6acf791331bdea5006cda105"
server_key_iv: "93b9ef17f83d413f72d81ad8afc8c962"
server_key_version: "encryptionkey@0"
I20260926 04:03:31.164106 9560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:31.185856 9560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:31.189491 9560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:31.189767 9560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:31.190143 9560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:31.190640 9560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:31.190845 9560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.190887 9560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:31.190950 9560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.199081 9560 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:33011
I20260926 04:03:31.199708 9560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:31.200603 9707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:33011 every 8 connection(s)
I20260926 04:03:31.205101 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9560
I20260926 04:03:31.205624 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:46251
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:45793
--webserver_port=42845
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.213121 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.201218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:44161)
0926 04:03:31.201539 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.201591 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.202141 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.203274 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.203278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.204177 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.204475 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.204481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.206198 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.206200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.207467 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.207470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.208122 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.208123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.208264 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.208885 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.208896 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.212824 (+ 3928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":90,"threads_started":1}
W20260926 04:03:31.213387 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.201534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44161 (local address 127.0.200.190:45793)
0926 04:03:31.202793 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.202802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.202839 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:03:31.202985 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.202992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.203049 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.203196 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.204612 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.206076 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.207577 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.208008 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.208145 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.213125 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:44161: BlockingRecv error: recv got EOF from 127.0.200.129:44161 (error 108)
Metrics: {"server-negotiator.queue_time_us":1205,"thread_start_us":113,"threads_started":1}
W20260926 04:03:31.214095 9708 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:45793 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.200.190:45793: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.221294 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.214465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:54563)
0926 04:03:31.214653 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.214670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.214771 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.215007 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.215008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.215219 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.215300 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.215303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.216476 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.216478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.217464 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.217468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.218097 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.218098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.218186 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.218771 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.218777 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.221213 (+ 2436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159}
W20260926 04:03:31.221436 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.214588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:54563 (local address 127.0.200.190:45793)
0926 04:03:31.214643 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.214646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.214767 (+ 121us) server_negotiation.cc:407] Connection header received
0926 04:03:31.214874 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.214875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.214894 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.214970 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.215424 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.216360 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.217616 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.218013 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.218086 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.221370 (+ 3284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:54563: BlockingRecv error: recv got EOF from 127.0.200.129:54563 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.227619 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.221867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:34161)
0926 04:03:31.221978 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.221989 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.222035 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.222354 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.222355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.222495 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.222592 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.222594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.223711 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.223712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.224420 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.224423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.224964 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.224965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.225028 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.225589 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.225593 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.227562 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260926 04:03:31.227746 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.221938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:34161 (local address 127.0.200.190:45793)
0926 04:03:31.221980 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.221982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.222021 (+ 39us) server_negotiation.cc:407] Connection header received
0926 04:03:31.222091 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.222092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.222103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.222329 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.222772 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.223599 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.224536 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.224867 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.224954 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.227680 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:34161: BlockingRecv error: recv got EOF from 127.0.200.129:34161 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 04:03:31.227801 9708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:03:31.299860 9712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:31.300092 9712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:31.300118 9712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:31.301374 9712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:31.301426 9712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:31.301457 9712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:31.303010 9712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.130.keytab
--principal=oryx/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:46251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=42845
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:31.303277 9712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:31.303560 9712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:31.310498 9718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:31.310554 9717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.310705 9712 server_base.cc:1034] running on GCE node
W20260926 04:03:31.310846 9720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.311089 9712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:31.311400 9712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:31.312615 9712 hybrid_clock.cc:648] HybridClock initialized: now 1790395411312619 us; error 46 us; skew 500 ppm
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395411, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:31.315364 9712 init.cc:377] Logged in from keytab as oryx/127.0.200.130@KRBTEST.COM (short username oryx)
I20260926 04:03:31.315881 9712 webserver.cc:460] Webserver started at http://127.0.200.130:42845/ using document root <none> and password file <none>
I20260926 04:03:31.316172 9712 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:31.316239 9712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:31.317940 9712 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:31.319171 9727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.319499 9712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:31.319635 9712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "cbd34e6c3e2044e5b7f27ae39e2b1aee"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "673a990d8a33edc2096987737029aa1a"
server_key_iv: "a26db65b300000db00b68785f875fd36"
server_key_version: "encryptionkey@0"
I20260926 04:03:31.320081 9712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:31.332537 9712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:31.335189 9712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:31.335379 9712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:31.335650 9712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:31.335980 9712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:31.336040 9712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.336086 9712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:31.336110 9712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.342310 9712 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:46251
I20260926 04:03:31.342411 9840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:46251 every 8 connection(s)
I20260926 04:03:31.342835 9712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
W20260926 04:03:31.352387 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.343856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:32797 (local address 127.0.200.190:45793)
0926 04:03:31.343940 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.343943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.344234 (+ 291us) server_negotiation.cc:407] Connection header received
0926 04:03:31.344925 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.344926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.344946 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.345071 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.346346 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.347155 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.348415 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.348629 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.348709 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.352329 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:32797: BlockingRecv error: recv got EOF from 127.0.200.130:32797 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 04:03:31.352514 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.343790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:32797)
0926 04:03:31.344013 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.344065 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.344775 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.345271 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.345277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.345941 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.346166 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.346172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.347288 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.347290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.348281 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.348284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.348781 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.348783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.348958 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.349698 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.349708 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.352227 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":134,"threads_started":1}
I20260926 04:03:31.352700 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9712
W20260926 04:03:31.352771 9841 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:45793 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.200.190:45793: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
I20260926 04:03:31.353089 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:44273
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:45793
--webserver_port=38543
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.358800 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.353097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:39521)
0926 04:03:31.353149 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.353158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.353218 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.353464 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.353465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.353611 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.353667 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.353668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.354660 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.354661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.355549 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.355552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.356066 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.356067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.356129 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.356665 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.356670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.358688 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 04:03:31.359040 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.353151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:39521 (local address 127.0.200.190:45793)
0926 04:03:31.353213 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.353215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.353265 (+ 50us) server_negotiation.cc:407] Connection header received
0926 04:03:31.353301 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.353302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.353314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.353384 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.353752 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.354525 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.355656 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.355961 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.356048 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.358832 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:39521: BlockingRecv error: recv got EOF from 127.0.200.130:39521 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.365160 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.359230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:44591)
0926 04:03:31.359290 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.359305 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.359359 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.359518 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.359520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.359695 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.359743 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.359746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.360908 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.360909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.361868 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.361872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.362347 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.362348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.362421 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.362951 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.362957 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.365079 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 04:03:31.365345 9841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:03:31.365393 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.359271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44591 (local address 127.0.200.190:45793)
0926 04:03:31.359331 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.359333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.359344 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:31.359429 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.359430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.359438 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.359486 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.359887 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.360696 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.362001 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.362221 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.362323 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.365326 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:44591: BlockingRecv error: recv got EOF from 127.0.200.130:44591 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 04:03:31.445945 9844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:31.446190 9844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:31.446219 9844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:31.447466 9844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:31.447523 9844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:31.447552 9844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:31.449095 9844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/krb5kdc/oryx_127.0.200.131.keytab
--principal=oryx/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:44273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=38543
--tserver_master_addrs=127.0.200.190:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:31.449333 9844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:31.449527 9844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:03:31.455610 9844 server_base.cc:1034] running on GCE node
W20260926 04:03:31.455397 9852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:31.455376 9849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:31.455371 9850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:31.456277 9844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:31.456619 9844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:31.457837 9844 hybrid_clock.cc:648] HybridClock initialized: now 1790395411457790 us; error 78 us; skew 500 ppm
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395411, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:31.460693 9844 init.cc:377] Logged in from keytab as oryx/127.0.200.131@KRBTEST.COM (short username oryx)
I20260926 04:03:31.461193 9844 webserver.cc:460] Webserver started at http://127.0.200.131:38543/ using document root <none> and password file <none>
I20260926 04:03:31.461411 9844 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:31.461477 9844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:31.463028 9844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:31.464087 9859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.464356 9844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:31.464483 9844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "8929dbe783ca41a79fd09678becf7f32"
format_stamp: "Formatted at 2026-09-26 04:03:30 on dist-test-slave-j8c1"
server_key: "15839aaa6072e153df25c8b71772b1f5"
server_key_iv: "42061b551b8a9be5381c8598460d2bd1"
server_key_version: "encryptionkey@0"
I20260926 04:03:31.464848 9844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:31.483348 9844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:31.485889 9844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:31.486023 9844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:31.486228 9844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:31.486660 9844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:31.486707 9844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.486732 9844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:31.486745 9844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:31.492821 9844 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:44273
I20260926 04:03:31.492918 9972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:44273 every 8 connection(s)
I20260926 04:03:31.493269 9844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:31.499600 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 9844
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.501588 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.494159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:50125)
0926 04:03:31.494336 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.494381 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.494920 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.495190 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.495193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.495774 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.495974 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.495978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.496715 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.496716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.497602 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.497606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.497974 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.497975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.498085 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.498764 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.498773 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.501428 (+ 2655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
W20260926 04:03:31.501788 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.494179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:50125 (local address 127.0.200.190:45793)
0926 04:03:31.494278 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.494281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.494637 (+ 356us) server_negotiation.cc:407] Connection header received
0926 04:03:31.495055 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.495056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.495066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.495134 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.496072 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.496608 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.497694 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.497889 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.497948 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.501687 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:50125: BlockingRecv error: recv got EOF from 127.0.200.131:50125 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 04:03:31.501967 9973 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:45793 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.200.190:45793: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.508172 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.502199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:36233)
0926 04:03:31.502240 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.502250 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.502287 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.502586 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.502587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.502736 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.502790 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.502792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.503648 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.503649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.504408 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.504411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.505101 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.505103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.505178 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.505587 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.505592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.508046 (+ 2454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260926 04:03:31.508322 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.502363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:36233 (local address 127.0.200.190:45793)
0926 04:03:31.502432 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.502434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.502443 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:31.502470 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.502470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.502478 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.502536 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.502866 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.503560 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.504512 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.504924 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.504992 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.508259 (+ 3267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:36233: BlockingRecv error: recv got EOF from 127.0.200.131:36233 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:03:31.511121 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.500273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.0.200.190:45793)
0926 04:03:31.500403 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.500405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.500414 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:31.500438 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.500439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.500452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.500535 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.501691 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.502759 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.503915 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.504222 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.504294 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.506231 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:31.506238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:31.506239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:31.506254 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:31.509177 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:31.509642 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:31.509646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:31.509647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:31.509772 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:31.510103 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:31.510105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:31.510106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:31.510476 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:31.510679 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:31.510864 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 04:03:31.511000 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:31 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:31.513912 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:31.508744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:41605)
0926 04:03:31.508849 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:31.508862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:31.508919 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:31.509121 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:31.509122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:31.509281 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:31.509326 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.509328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.510131 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.510133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.510963 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:31.510966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.511345 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.511346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:31.511403 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.511890 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:31.511895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:31.513850 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260926 04:03:31.514114 9711 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:31.508936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:41605 (local address 127.0.200.190:45793)
0926 04:03:31.509008 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:31.509010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:31.509018 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:31.509038 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:31.509038 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:31.509047 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:31.509095 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:31.509408 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.510043 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.511051 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:31.511267 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:31.511342 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:31.513946 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:41605: BlockingRecv error: recv got EOF from 127.0.200.131:41605 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 04:03:31.514226 9973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:32.233429 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:32.228273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:58195)
0926 04:03:32.228585 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:32.228596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:32.228666 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:32.228842 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:32.228843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:32.229019 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:32.229067 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.229069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.229793 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.229794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.230350 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.230351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.230750 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.230751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.230794 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.231356 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:32.231361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:32.233363 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":240,"threads_started":1}
W20260926 04:03:32.233625 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:32.228501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:58195 (local address 127.0.200.190:45793)
0926 04:03:32.228611 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:32.228613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:32.228665 (+ 52us) server_negotiation.cc:407] Connection header received
0926 04:03:32.228724 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:32.228725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:32.228735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:32.228787 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:32.229157 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.229703 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.230464 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.230682 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.230740 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.233527 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:58195: BlockingRecv error: recv got EOF from 127.0.200.129:58195 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":55,"threads_started":1}
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:32.371142 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:32.365788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:37649)
0926 04:03:32.365996 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:32.366008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:32.366071 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:32.366263 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:32.366264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:32.366439 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:32.366487 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.366489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.367341 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.367342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.368043 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.368046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.368491 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.368492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.368548 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.369137 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:32.369142 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:32.371076 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
W20260926 04:03:32.371246 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:32.365972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:37649 (local address 127.0.200.190:45793)
0926 04:03:32.366041 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:32.366045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:32.366060 (+ 15us) server_negotiation.cc:407] Connection header received
0926 04:03:32.366174 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:32.366175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:32.366186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:32.366247 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:32.366579 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.367229 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.368128 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.368309 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.368384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.371178 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:37649: BlockingRecv error: recv got EOF from 127.0.200.130:37649 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:32 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:32.519742 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:32.514737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:33995)
0926 04:03:32.514947 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:32.514959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:32.515063 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:32.515248 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:32.515248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:32.515445 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:32.515514 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.515516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.516221 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.516222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.516781 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:32.516783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.517159 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.517160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:32.517236 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.517772 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:32.517776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:32.519680 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
W20260926 04:03:32.519883 9979 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:32.514856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33995 (local address 127.0.200.190:45793)
0926 04:03:32.514928 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:32.514930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:32.515042 (+ 112us) server_negotiation.cc:407] Connection header received
0926 04:03:32.515147 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:32.515148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:32.515164 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:32.515238 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:32.515630 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.516121 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.516875 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:32.517072 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:32.517157 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:32.519786 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:33995: BlockingRecv error: recv got EOF from 127.0.200.131:33995 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:33.238729 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:33.234017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:55001)
0926 04:03:33.234160 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:33.234173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:33.234234 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:33.234425 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:33.234425 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:33.234701 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:33.234749 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.234751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.235394 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.235395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.235950 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.235952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.236341 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.236341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.236380 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.236848 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:33.236852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:33.238637 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
W20260926 04:03:33.238785 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:33.234097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55001 (local address 127.0.200.190:45793)
0926 04:03:33.234232 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:33.234234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:33.234242 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:33.234316 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:33.234317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:33.234332 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:33.234421 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:33.234823 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.235321 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.236073 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.236264 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.236330 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.238731 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:55001: BlockingRecv error: recv got EOF from 127.0.200.129:55001 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:33.376444 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:33.371696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:45505)
0926 04:03:33.371868 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:33.371879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:33.371941 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:33.372069 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:33.372070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:33.372234 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:33.372283 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.372284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.372943 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.372943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.373455 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.373456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.373750 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.373751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.373801 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.374378 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:33.374383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:33.376386 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
W20260926 04:03:33.376575 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:33.371741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:45505 (local address 127.0.200.190:45793)
0926 04:03:33.371813 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:33.371816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:33.371942 (+ 126us) server_negotiation.cc:407] Connection header received
0926 04:03:33.371997 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:33.371997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:33.372007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:33.372049 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:33.372348 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.372871 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.373519 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.373693 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.373740 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.376497 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:45505: BlockingRecv error: recv got EOF from 127.0.200.130:45505 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:33 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:33.524978 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:33.520269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:44487)
0926 04:03:33.520452 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:33.520466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:33.520541 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:33.520666 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:33.520666 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:33.520823 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:33.520870 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.520872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.521520 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.521521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.522048 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:33.522050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.522375 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.522377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:33.522431 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.522917 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:33.522921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:33.524882 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
W20260926 04:03:33.525015 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:33.520437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:44487 (local address 127.0.200.190:45793)
0926 04:03:33.520501 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:33.520503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:33.520515 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:33.520590 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:33.520591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:33.520603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:33.520649 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:33.520930 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.521424 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.522119 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:33.522299 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:33.522347 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:33.524956 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:44487: BlockingRecv error: recv got EOF from 127.0.200.131:44487 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:34.244665 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:34.239325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:39281)
0926 04:03:34.239503 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:34.239517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:34.239610 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:34.239851 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:34.239852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:34.240014 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:34.240061 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.240063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.240947 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.240948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.241485 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.241487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.241881 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.241882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.241940 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.242517 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:34.242522 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:34.244587 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
W20260926 04:03:34.244833 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:34.239447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:39281 (local address 127.0.200.190:45793)
0926 04:03:34.239562 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:34.239564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:34.239592 (+ 28us) server_negotiation.cc:407] Connection header received
0926 04:03:34.239668 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:34.239669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:34.239679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:34.239744 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:34.240165 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.240845 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.241576 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.241804 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.241884 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.244723 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:39281: BlockingRecv error: recv got EOF from 127.0.200.129:39281 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:34.383498 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:34.376909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:43559)
0926 04:03:34.377023 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:34.377034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:34.377106 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:34.377278 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:34.377279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:34.377504 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:34.377571 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.377573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.378653 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.378654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.379620 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.379623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.380092 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.380093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.380187 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.380752 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:34.380759 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:34.383372 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
W20260926 04:03:34.383553 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:34.376951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43559 (local address 127.0.200.190:45793)
0926 04:03:34.376995 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:34.376997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:34.377090 (+ 93us) server_negotiation.cc:407] Connection header received
0926 04:03:34.377175 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:34.377175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:34.377187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:34.377269 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:34.377644 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.378515 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.379715 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.380031 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.380089 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.383484 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:43559: BlockingRecv error: recv got EOF from 127.0.200.130:43559 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:34 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:34.530380 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:34.525486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:49049)
0926 04:03:34.525738 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:34.525751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:34.525848 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:34.526014 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:34.526015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:34.526196 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:34.526243 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.526245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.526981 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.526982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.527527 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:34.527529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.527887 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.527888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:34.527951 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.528530 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:34.528534 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:34.530269 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
W20260926 04:03:34.530409 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:34.525584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:49049 (local address 127.0.200.190:45793)
0926 04:03:34.525687 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:34.525690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:34.525832 (+ 142us) server_negotiation.cc:407] Connection header received
0926 04:03:34.525918 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:34.525919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:34.525940 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:34.526005 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:34.526323 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.526899 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.527612 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:34.527800 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:34.527885 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:34.530345 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:49049: BlockingRecv error: recv got EOF from 127.0.200.131:49049 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:35.257905 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:35.245216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:34553)
0926 04:03:35.245435 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:35.245454 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:35.245523 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:35.245686 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:35.245687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:35.245854 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:35.245898 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.245900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.246669 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.246671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.247273 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.247275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.247633 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.247633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.247709 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.256095 (+ 8386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:35.256101 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:35.257845 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":138,"threads_started":1}
W20260926 04:03:35.258080 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:35.245309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:34553 (local address 127.0.200.190:45793)
0926 04:03:35.245433 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:35.245435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:35.245528 (+ 93us) server_negotiation.cc:407] Connection header received
0926 04:03:35.245569 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:35.245570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:35.245581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:35.245640 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:35.246021 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.246527 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.247353 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.247564 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.247622 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.258005 (+ 10383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:34553: BlockingRecv error: recv got EOF from 127.0.200.129:34553 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:35.388736 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:35.384029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:37807)
0926 04:03:35.384174 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:35.384186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:35.384276 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:35.384458 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:35.384459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:35.384653 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:35.384698 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.384700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.385425 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.385427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.385948 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.385950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.386335 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.386335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.386383 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.386869 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:35.386874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:35.388665 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
W20260926 04:03:35.388866 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:35.384070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:37807 (local address 127.0.200.190:45793)
0926 04:03:35.384127 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:35.384130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:35.384264 (+ 134us) server_negotiation.cc:407] Connection header received
0926 04:03:35.384344 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:35.384345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:35.384359 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:35.384450 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:35.384789 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.385329 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.386057 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.386246 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.386318 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.388782 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:37807: BlockingRecv error: recv got EOF from 127.0.200.130:37807 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:35 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:35.535986 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:35.530952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:50773)
0926 04:03:35.531095 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:35.531107 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:35.531195 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:35.531381 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:35.531382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:35.531560 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:35.531604 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.531606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.532263 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.532264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.532790 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:35.532792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.533266 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.533267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:35.533337 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.533885 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:35.533890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:35.535859 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
W20260926 04:03:35.536006 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:35.531094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:50773 (local address 127.0.200.190:45793)
0926 04:03:35.531169 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:35.531171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:35.531181 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:35.531267 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:35.531268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:35.531300 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:35.531372 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:35.531684 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.532179 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.532877 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:35.533149 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:35.533263 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:35.535951 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:50773: BlockingRecv error: recv got EOF from 127.0.200.131:50773 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:36.263427 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:36.258424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:55355)
0926 04:03:36.258626 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:36.258645 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:36.258708 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:36.258880 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:36.258880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:36.259102 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:36.259150 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.259152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.259968 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.259969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.260503 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.260505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.260925 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.260926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.260970 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.261429 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:36.261434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:36.263365 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
W20260926 04:03:36.263536 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:36.258576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55355 (local address 127.0.200.190:45793)
0926 04:03:36.258696 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:36.258697 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:36.258707 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:36.258768 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:36.258769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:36.258797 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:36.258863 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:36.259249 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.259858 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.260626 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.260820 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.260886 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.263456 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:55355: BlockingRecv error: recv got EOF from 127.0.200.129:55355 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:36.394302 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:36.389391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:35613)
0926 04:03:36.389555 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:36.389573 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:36.389652 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:36.389813 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:36.389814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:36.389996 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:36.390040 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.390042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.390779 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.390780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.391308 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.391310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.391671 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.391672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.391726 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.392274 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:36.392284 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:36.394242 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
W20260926 04:03:36.394454 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:36.389583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:35613 (local address 127.0.200.190:45793)
0926 04:03:36.389658 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:36.389660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:36.389689 (+ 29us) server_negotiation.cc:407] Connection header received
0926 04:03:36.389721 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:36.389722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:36.389740 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:36.389804 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:36.390121 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.390698 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.391395 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.391596 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.391674 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.394348 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:35613: BlockingRecv error: recv got EOF from 127.0.200.130:35613 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:36 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:36.541090 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:36.536496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:40799)
0926 04:03:36.536649 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:36.536661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:36.536726 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:36.536861 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:36.536861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:36.537023 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:36.537103 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.537105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.537741 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.537741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.538282 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:36.538283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.538688 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.538689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:36.538726 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.539253 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:36.539258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:36.541027 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
W20260926 04:03:36.541213 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:36.536554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:40799 (local address 127.0.200.190:45793)
0926 04:03:36.536619 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:36.536621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:36.536748 (+ 127us) server_negotiation.cc:407] Connection header received
0926 04:03:36.536774 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:36.536774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:36.536787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:36.536843 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:36.537177 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.537674 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.538359 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:36.538614 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:36.538681 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:36.541132 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:40799: BlockingRecv error: recv got EOF from 127.0.200.131:40799 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:37.269381 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:37.263970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:57395)
0926 04:03:37.264187 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:37.264198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:37.264264 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:37.264490 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:37.264491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:37.264662 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:37.264712 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.264714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.265467 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.265468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.266046 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.266048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.266467 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.266468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.266516 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.267104 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:37.267109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:37.269262 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":151,"threads_started":1}
W20260926 04:03:37.269487 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:37.264075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:57395 (local address 127.0.200.190:45793)
0926 04:03:37.264208 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:37.264210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:37.264326 (+ 116us) server_negotiation.cc:407] Connection header received
0926 04:03:37.264357 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:37.264357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:37.264369 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:37.264435 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:37.264803 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.265350 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.266154 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.266379 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.266444 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.269405 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:57395: BlockingRecv error: recv got EOF from 127.0.200.129:57395 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":53,"threads_started":1}
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:37.399534 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:37.394908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:48851)
0926 04:03:37.395062 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:37.395075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:37.395129 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:37.395263 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:37.395264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:37.395441 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:37.395488 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.395490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.396150 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.396150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.396704 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.396706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.397080 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.397081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.397150 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.397601 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:37.397606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:37.399452 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":112,"threads_started":1}
W20260926 04:03:37.399626 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:37.395037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:48851 (local address 127.0.200.190:45793)
0926 04:03:37.395105 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:37.395107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:37.395128 (+ 21us) server_negotiation.cc:407] Connection header received
0926 04:03:37.395192 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:37.395192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:37.395205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:37.395248 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:37.395567 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.396062 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.396796 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.396996 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.397079 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.399562 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:48851: BlockingRecv error: recv got EOF from 127.0.200.130:48851 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:37 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:37.546480 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:37.541668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:37855)
0926 04:03:37.541827 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:37.541838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:37.541903 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:37.542031 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:37.542032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:37.542195 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:37.542245 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.542247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.542934 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.542934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.543475 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:37.543477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.543852 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.543853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:37.543891 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.544359 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:37.544363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:37.546419 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
W20260926 04:03:37.546712 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:37.541726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:37855 (local address 127.0.200.190:45793)
0926 04:03:37.541797 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:37.541799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:37.541903 (+ 104us) server_negotiation.cc:407] Connection header received
0926 04:03:37.541956 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:37.541957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:37.541968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:37.542011 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:37.542346 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.542863 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.543570 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:37.543775 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:37.543855 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:37.546649 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:37855: BlockingRecv error: recv got EOF from 127.0.200.131:37855 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:38.275136 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:38.269940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:38289)
0926 04:03:38.270110 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:38.270129 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:38.270194 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:38.270349 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:38.270350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:38.270526 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:38.270661 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.270663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.271638 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.271639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.272186 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.272188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.272589 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.272590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.272651 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.273199 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:38.273204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:38.275036 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
W20260926 04:03:38.275187 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:38.269985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:38289 (local address 127.0.200.190:45793)
0926 04:03:38.270167 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:38.270169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:38.270181 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:03:38.270261 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:38.270262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:38.270272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:38.270324 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:38.270757 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.271501 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.272276 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.272500 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.272589 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.275117 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:38289: BlockingRecv error: recv got EOF from 127.0.200.129:38289 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:38.405150 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:38.400355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:42117)
0926 04:03:38.400519 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:38.400538 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:38.400598 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:38.400778 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:38.400779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:38.400950 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:38.401000 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.401002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.401659 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.401660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.402196 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.402198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.402541 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.402542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.402649 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.403163 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:38.403169 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:38.405091 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
W20260926 04:03:38.405369 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:38.400441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:42117 (local address 127.0.200.190:45793)
0926 04:03:38.400520 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:38.400522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:38.400603 (+ 81us) server_negotiation.cc:407] Connection header received
0926 04:03:38.400669 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:38.400670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:38.400681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:38.400744 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:38.401072 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.401581 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.402285 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.402470 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.402529 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.405298 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:42117: BlockingRecv error: recv got EOF from 127.0.200.130:42117 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:38 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:38.552404 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:38.547090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:38107)
0926 04:03:38.547247 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:38.547263 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:38.547432 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:38.547721 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:38.547722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:38.547890 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:38.547938 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.547939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.548677 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.548678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.549423 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:38.549426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.549801 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.549802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:38.549864 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.550445 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:38.550450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:38.552344 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
W20260926 04:03:38.552556 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:38.547207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:38107 (local address 127.0.200.190:45793)
0926 04:03:38.547285 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:38.547287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:38.547318 (+ 31us) server_negotiation.cc:407] Connection header received
0926 04:03:38.547603 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:38.547605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:38.547617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:38.547674 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:38.548014 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.548556 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.549509 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:38.549712 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:38.549780 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:38.552472 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:38107: BlockingRecv error: recv got EOF from 127.0.200.131:38107 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:39.280591 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:39.275749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:41623)
0926 04:03:39.275903 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:39.275915 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:39.275979 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:39.276259 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:39.276261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:39.276427 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:39.276473 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.276475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.277191 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.277192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.277709 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.277710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.278071 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.278072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.278113 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.278643 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:39.278647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:39.280530 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260926 04:03:39.280803 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:39.275938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:41623 (local address 127.0.200.190:45793)
0926 04:03:39.276076 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:39.276078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:39.276086 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:39.276111 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:39.276112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:39.276122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:39.276196 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:39.276599 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.277095 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.277806 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.277999 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.278060 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.280712 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:41623: BlockingRecv error: recv got EOF from 127.0.200.129:41623 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:39.410748 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:39.405726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:44237)
0926 04:03:39.405904 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:39.405913 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:39.405988 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:39.406137 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:39.406137 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:39.406319 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:39.406362 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.406364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.407295 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.407297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.407843 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.407844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.408206 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.408207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.408260 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.408782 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:39.408787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:39.410657 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
W20260926 04:03:39.410825 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:39.405780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44237 (local address 127.0.200.190:45793)
0926 04:03:39.405850 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:39.405852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:39.405974 (+ 122us) server_negotiation.cc:407] Connection header received
0926 04:03:39.406045 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:39.406046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:39.406059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:39.406131 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:39.406443 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.407140 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.407939 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.408125 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.408212 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.410759 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:44237: BlockingRecv error: recv got EOF from 127.0.200.130:44237 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:39 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:39.557699 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:39.552876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:48515)
0926 04:03:39.553033 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:39.553047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:39.553124 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:39.553287 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:39.553288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:39.553476 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:39.553521 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.553523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.554202 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.554204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.554827 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:39.554829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.555206 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.555207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:39.555262 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.555739 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:39.555743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:39.557615 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260926 04:03:39.557822 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:39.553000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:48515 (local address 127.0.200.190:45793)
0926 04:03:39.553047 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:39.553049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:39.553103 (+ 54us) server_negotiation.cc:407] Connection header received
0926 04:03:39.553188 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:39.553189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:39.553205 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:39.553278 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:39.553613 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.554118 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.554929 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:39.555120 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:39.555207 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:39.557739 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:48515: BlockingRecv error: recv got EOF from 127.0.200.131:48515 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:40.293965 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:40.281184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:55615)
0926 04:03:40.281459 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:40.281471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:40.281548 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:40.281755 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:40.281756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:40.281925 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:40.281971 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.281972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.282814 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.282815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.283427 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.283429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.283778 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.283779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.283850 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.291653 (+ 7803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:40.291660 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:40.293869 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":159,"threads_started":1}
W20260926 04:03:40.294075 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:40.281327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55615 (local address 127.0.200.190:45793)
0926 04:03:40.281464 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:40.281466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:40.281594 (+ 128us) server_negotiation.cc:407] Connection header received
0926 04:03:40.281626 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:40.281627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:40.281638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:40.281700 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:40.282083 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.282718 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.283523 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.283708 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.283758 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.293993 (+ 10235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:55615: BlockingRecv error: recv got EOF from 127.0.200.129:55615 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:40.416041 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:40.411256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:42407)
0926 04:03:40.411415 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:40.411426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:40.411474 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:40.411611 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:40.411611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:40.411769 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:40.411816 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.411817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.412450 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.412450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.413001 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.413002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.413326 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.413326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.413364 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.413866 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:40.413871 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:40.415922 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260926 04:03:40.416149 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:40.411368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:42407 (local address 127.0.200.190:45793)
0926 04:03:40.411467 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:40.411468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:40.411477 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:40.411523 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:40.411524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:40.411534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:40.411591 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:40.411886 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.412375 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.413083 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.413260 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.413313 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.416048 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:42407: BlockingRecv error: recv got EOF from 127.0.200.130:42407 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:40 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:40.564178 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:40.558378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:60513)
0926 04:03:40.558545 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:40.558593 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:40.558660 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:40.558927 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:40.558929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:40.559099 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:40.559147 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.559149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.559929 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.559930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.560455 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:40.560457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.560860 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.560861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:40.560908 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.561407 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:40.561412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:40.564076 (+ 2664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
W20260926 04:03:40.564244 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:40.558526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:60513 (local address 127.0.200.190:45793)
0926 04:03:40.558638 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:40.558640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:40.558651 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:40.558779 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:40.558780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:40.558794 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:40.558866 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:40.559289 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.559812 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.560575 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:40.560751 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:40.560821 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:40.564177 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:60513: BlockingRecv error: recv got EOF from 127.0.200.131:60513 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:41.299458 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:41.294677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:55897)
0926 04:03:41.294802 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:41.294813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:41.294891 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:41.295111 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:41.295112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:41.295325 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:41.295380 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.295382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.296037 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.296038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.296586 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.296588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.296926 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.296927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.296973 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.297483 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:41.297489 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:41.299398 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
W20260926 04:03:41.299605 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:41.294690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55897 (local address 127.0.200.190:45793)
0926 04:03:41.294836 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:41.294839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:41.294892 (+ 53us) server_negotiation.cc:407] Connection header received
0926 04:03:41.294981 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:41.294981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:41.294994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:41.295093 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:41.295435 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.295926 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.296669 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.296856 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.296911 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.299514 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:55897: BlockingRecv error: recv got EOF from 127.0.200.129:55897 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:41.421629 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:41.416550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:54971)
0926 04:03:41.416699 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:41.416715 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:41.416773 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:41.416953 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:41.416954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:41.417121 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:41.417189 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.417191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.418003 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.418004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.418581 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.418583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.418903 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.418904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.418947 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.419493 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:41.419498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:41.421538 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260926 04:03:41.421717 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:41.416663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:54971 (local address 127.0.200.190:45793)
0926 04:03:41.416718 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:41.416721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:41.416758 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:03:41.416833 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:41.416834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:41.416845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:41.416911 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:41.417287 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.417905 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.418648 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.418835 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.418892 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.421656 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:54971: BlockingRecv error: recv got EOF from 127.0.200.130:54971 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:41 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:41.570079 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:41.564823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:34061)
0926 04:03:41.565018 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:41.565030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:41.565082 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:41.565246 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:41.565247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:41.565415 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:41.565461 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.565462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.566413 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.566414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.566979 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:41.566981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.567360 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.567361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:41.567404 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.567920 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:41.567925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:41.570003 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
W20260926 04:03:41.570204 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:41.565023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:34061 (local address 127.0.200.190:45793)
0926 04:03:41.565098 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:41.565100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:41.565109 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:41.565135 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:41.565136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:41.565147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:41.565212 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:41.565564 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.566190 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.567066 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:41.567285 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:41.567347 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:41.570115 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:34061: BlockingRecv error: recv got EOF from 127.0.200.131:34061 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:42.304772 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:42.299930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:57279)
0926 04:03:42.300095 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:42.300109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:42.300213 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:42.300350 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:42.300350 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:42.300521 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:42.300570 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.300571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.301266 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.301267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.301816 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.301818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.302180 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.302181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.302226 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.302763 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:42.302768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:42.304704 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
W20260926 04:03:42.304888 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:42.300039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:57279 (local address 127.0.200.190:45793)
0926 04:03:42.300170 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:42.300172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:42.300186 (+ 14us) server_negotiation.cc:407] Connection header received
0926 04:03:42.300270 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:42.300271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:42.300281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:42.300335 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:42.300668 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.301185 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.301901 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.302087 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.302164 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.304806 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:57279: BlockingRecv error: recv got EOF from 127.0.200.129:57279 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":105,"threads_started":1}
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:42.427608 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:42.422309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:37073)
0926 04:03:42.422478 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:42.422490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:42.422541 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:42.422730 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:42.422730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:42.422899 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:42.422946 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.422948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.423649 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.423650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.424242 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.424244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.424700 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.424701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.424750 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.425241 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:42.425245 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:42.427535 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
W20260926 04:03:42.427816 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:42.422399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:37073 (local address 127.0.200.190:45793)
0926 04:03:42.422517 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:42.422519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:42.422563 (+ 44us) server_negotiation.cc:407] Connection header received
0926 04:03:42.422659 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:42.422659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:42.422670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:42.422714 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:42.423007 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.423527 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.424378 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.424570 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.424637 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.427756 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:37073: BlockingRecv error: recv got EOF from 127.0.200.130:37073 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:42 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:42.575453 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:42.570678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:54339)
0926 04:03:42.570937 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:42.570949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:42.571012 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:42.571174 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:42.571175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:42.571348 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:42.571395 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.571396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.572063 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.572064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.572613 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:42.572615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.573000 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.573001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:42.573048 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.573600 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:42.573605 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:42.575388 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":71,"threads_started":1}
W20260926 04:03:42.575601 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:42.570874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:54339 (local address 127.0.200.190:45793)
0926 04:03:42.570952 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:42.570954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:42.571008 (+ 54us) server_negotiation.cc:407] Connection header received
0926 04:03:42.571055 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:42.571056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:42.571069 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:42.571129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:42.571476 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.571979 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.572713 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:42.572910 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:42.572972 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:42.575518 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:54339: BlockingRecv error: recv got EOF from 127.0.200.131:54339 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:43.310246 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:43.305356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:60975)
0926 04:03:43.305555 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:43.305568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:43.305651 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:43.305791 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:43.305792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:43.305977 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:43.306021 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.306023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.306764 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.306765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.307305 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.307306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.307655 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.307655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.307693 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.308285 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:43.308291 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:43.310180 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":65,"threads_started":1}
W20260926 04:03:43.310406 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:43.305504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:60975 (local address 127.0.200.190:45793)
0926 04:03:43.305651 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:43.305653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:43.305662 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:43.305696 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:43.305696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:43.305707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:43.305768 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:43.306098 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.306681 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.307398 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.307587 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.307644 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.310299 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:60975: BlockingRecv error: recv got EOF from 127.0.200.129:60975 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:43.433379 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:43.428218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:33367)
0926 04:03:43.428428 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:43.428452 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:43.428545 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:43.428730 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:43.428731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:43.428917 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:43.428961 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.428963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.429652 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.429653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.430601 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.430603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.430965 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.430966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.431021 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.431540 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:43.431544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:43.433320 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":132,"threads_started":1}
W20260926 04:03:43.433461 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:43.428302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:33367 (local address 127.0.200.190:45793)
0926 04:03:43.428397 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:43.428399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:43.428524 (+ 125us) server_negotiation.cc:407] Connection header received
0926 04:03:43.428646 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:43.428646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:43.428657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:43.428725 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:43.429043 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.429552 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.430685 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.430877 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.430967 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.433414 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:33367: BlockingRecv error: recv got EOF from 127.0.200.130:33367 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:43 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:43.580555 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:43.575979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:33077)
0926 04:03:43.576133 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:43.576144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:43.576205 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:43.576343 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:43.576344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:43.576508 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:43.576550 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.576551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.577258 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.577259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.577812 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:43.577814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.578152 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.578153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:43.578192 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.578753 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:43.578758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:43.580496 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":107,"threads_started":1}
W20260926 04:03:43.580705 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:43.576043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33077 (local address 127.0.200.190:45793)
0926 04:03:43.576110 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:43.576112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:43.576216 (+ 104us) server_negotiation.cc:407] Connection header received
0926 04:03:43.576244 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:43.576244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:43.576257 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:43.576317 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:43.576662 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.577178 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.577900 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:43.578080 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:43.578144 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:43.580597 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:33077: BlockingRecv error: recv got EOF from 127.0.200.131:33077 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:44.322060 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:44.310884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:37335)
0926 04:03:44.311073 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:44.311084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:44.311141 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:44.311283 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:44.311284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:44.311459 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:44.311505 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.311507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.312271 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.312272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.312823 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.312825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.313260 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.313261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.313306 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.319432 (+ 6126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:44.319440 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:44.321997 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":114,"threads_started":1}
W20260926 04:03:44.322278 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:44.311003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:37335 (local address 127.0.200.190:45793)
0926 04:03:44.311128 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:44.311130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:44.311139 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:03:44.311198 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:44.311199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:44.311209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:44.311268 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:44.311617 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.312181 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.312956 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.313171 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.313249 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.322194 (+ 8945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:37335: BlockingRecv error: recv got EOF from 127.0.200.129:37335 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:44.439459 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:44.433989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:43085)
0926 04:03:44.434133 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:44.434145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:44.434209 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:44.434379 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:44.434380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:44.434633 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:44.434690 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.434692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.435374 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.435376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.435929 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.435931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.436388 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.436390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.436443 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.437035 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:44.437040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:44.439333 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
W20260926 04:03:44.439611 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:44.434088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43085 (local address 127.0.200.190:45793)
0926 04:03:44.434140 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:44.434143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:44.434188 (+ 45us) server_negotiation.cc:407] Connection header received
0926 04:03:44.434286 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:44.434287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:44.434298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:44.434362 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:44.434766 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.435272 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.436019 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.436313 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.436391 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.439502 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:43085: BlockingRecv error: recv got EOF from 127.0.200.130:43085 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:44 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:44.586498 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:44.581168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:33911)
0926 04:03:44.581312 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:44.581326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:44.581449 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:44.581616 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:44.581617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:44.581813 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:44.581858 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.581860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.582541 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.582570 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.583109 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:44.583112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.583501 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.583502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:44.583557 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.584107 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:44.584112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:44.586376 (+ 2264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260926 04:03:44.586620 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:44.581287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33911 (local address 127.0.200.190:45793)
0926 04:03:44.581340 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:44.581342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:44.581429 (+ 87us) server_negotiation.cc:407] Connection header received
0926 04:03:44.581522 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:44.581522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:44.581536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:44.581608 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:44.581952 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.582443 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.583203 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:44.583394 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:44.583500 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:44.586510 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:33911: BlockingRecv error: recv got EOF from 127.0.200.131:33911 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:45.332772 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:45.322525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:40187)
0926 04:03:45.322682 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:45.322694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:45.322755 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:45.322930 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:45.322930 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:45.323149 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:45.323195 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.323197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.323910 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.323911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.324546 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.324548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.324887 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.324888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.324927 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.330471 (+ 5544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:45.330478 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:45.332694 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
W20260926 04:03:45.333038 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:45.322648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:40187 (local address 127.0.200.190:45793)
0926 04:03:45.322777 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:45.322779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:45.322787 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:03:45.322832 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:45.322832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:45.322843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:45.322905 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:45.323319 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.323835 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.324631 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.324821 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.324877 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.332985 (+ 8108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:40187: BlockingRecv error: recv got EOF from 127.0.200.129:40187 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:45.444743 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:45.440106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:48767)
0926 04:03:45.440263 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:45.440278 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:45.440335 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:45.440478 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:45.440478 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:45.440658 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:45.440705 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.440708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.441391 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.441392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.441916 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.441918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.442242 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.442243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.442284 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.442907 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:45.442912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:45.444682 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
W20260926 04:03:45.444916 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:45.440197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:48767 (local address 127.0.200.190:45793)
0926 04:03:45.440275 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:45.440277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:45.440343 (+ 66us) server_negotiation.cc:407] Connection header received
0926 04:03:45.440399 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:45.440400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:45.440411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:45.440457 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:45.440793 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.441301 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.441994 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.442175 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.442233 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.444798 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:48767: BlockingRecv error: recv got EOF from 127.0.200.130:48767 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:45 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.131@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:45.592032 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:45.586998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.131:38915)
0926 04:03:45.587146 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:45.587159 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:45.587224 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:45.587368 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:45.587369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:45.587538 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:45.587582 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.587584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.588320 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.588321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.588898 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:45.588900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.589352 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.589353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:45.589402 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.589986 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:45.589991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:45.591916 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":123,"threads_started":1}
W20260926 04:03:45.592077 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:45.587124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:38915 (local address 127.0.200.190:45793)
0926 04:03:45.587188 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:45.587190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:45.587212 (+ 22us) server_negotiation.cc:407] Connection header received
0926 04:03:45.587281 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:45.587281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:45.587292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:45.587346 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:45.587679 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.588230 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.589030 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:45.589230 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:45.589294 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:45.592009 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.131:38915: BlockingRecv error: recv got EOF from 127.0.200.131:38915 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 26 04:03:46 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:46 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.129@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:46.338140 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.333316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.129:57223)
0926 04:03:46.333463 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:46.333478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:46.333527 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:46.333686 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:46.333687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:46.333843 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:46.333888 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.333890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.334650 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.334651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.335198 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.335200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.335531 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.335532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.335574 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.336121 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:46.336133 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:46.338070 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
W20260926 04:03:46.338320 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:46.333395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:57223 (local address 127.0.200.190:45793)
0926 04:03:46.333546 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:46.333548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:46.333558 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:46.333589 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:46.333589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:46.333600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:46.333665 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:46.333979 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.334513 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.335278 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.335464 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.335515 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.338248 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.129:57223: BlockingRecv error: recv got EOF from 127.0.200.129:57223 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":101,"threads_started":1}
Sep 26 04:03:46 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
Sep 26 04:03:46 dist-test-slave-j8c1 krb5kdc[8977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.200.130@KRBTEST.COM for oryx/127.0.200.190@KRBTEST.COM, Server not found in Kerberos database
I20260926 04:03:46.450503 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.445494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45793 (local address 127.0.200.130:60027)
0926 04:03:46.445676 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:46.445686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:46.445747 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:46.445891 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:46.445891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:46.446083 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:46.446131 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.446133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.446870 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.446871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.447412 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.447413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.447803 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.447804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.447851 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.448398 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:46.448402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:46.450391 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.200.190:45793: Server oryx/127.0.200.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":105,"threads_started":1}
W20260926 04:03:46.450534 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:46.445580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60027 (local address 127.0.200.190:45793)
0926 04:03:46.445675 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:46.445677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:46.445752 (+ 75us) server_negotiation.cc:407] Connection header received
0926 04:03:46.445797 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:46.445798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:46.445810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:46.445873 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:46.446235 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.446777 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.447492 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.447674 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.447758 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.450470 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.200.130:60027: BlockingRecv error: recv got EOF from 127.0.200.130:60027 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 04:03:46.504751 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9560
I20260926 04:03:46.509624 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9712
I20260926 04:03:46.513993 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9844
I20260926 04:03:46.518158 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 9490
2026-09-26T04:03:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17805 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T04:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:46Z Disabled control of system clock
I20260926 04:03:46.546751 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:42275
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:45593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:42275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 04:03:46.628715 10040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:46.628947 10040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:46.628968 10040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:46.630234 10040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:46.630285 10040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:46.630306 10040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:46.630319 10040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:46.630332 10040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:46.631875 10040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:42275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:42275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:46.632167 10040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:46.632489 10040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:46.637861 10048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:46.637897 10040 server_base.cc:1034] running on GCE node
W20260926 04:03:46.637861 10046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:46.637876 10045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:46.638581 10040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:46.638923 10040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:46.640143 10040 hybrid_clock.cc:648] HybridClock initialized: now 1790395426640154 us; error 55 us; skew 500 ppm
I20260926 04:03:46.641554 10040 webserver.cc:460] Webserver started at http://127.0.200.190:46655/ using document root <none> and password file <none>
I20260926 04:03:46.641822 10040 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:46.641897 10040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:46.642048 10040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:46.643224 10040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "254d4aad57d545af94e69ea6a31d304c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "097710d3fce35c3745bbe0ddc98dd2c8"
server_key_iv: "26e8e97fe0feb5f1469837b32b3592b3"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.643605 10040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "254d4aad57d545af94e69ea6a31d304c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "097710d3fce35c3745bbe0ddc98dd2c8"
server_key_iv: "26e8e97fe0feb5f1469837b32b3592b3"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.645066 10040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:46.645977 10054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.646230 10040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:46.646344 10040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "254d4aad57d545af94e69ea6a31d304c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "097710d3fce35c3745bbe0ddc98dd2c8"
server_key_iv: "26e8e97fe0feb5f1469837b32b3592b3"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.646427 10040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:46.663763 10040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:46.664134 10040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:46.664304 10040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:46.669018 10040 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:42275
I20260926 04:03:46.669070 10106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:42275 every 8 connection(s)
I20260926 04:03:46.669453 10040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:46.670450 10107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:46.671444 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10040
I20260926 04:03:46.671543 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:46.671741 802 external_mini_cluster.cc:1428] Setting key 235d3af9d6c9761d6f91caf7e3a7f8e2
I20260926 04:03:46.673873 10107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c: Bootstrap starting.
I20260926 04:03:46.674501 10107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:46.674870 10107 log.cc:824] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:46.675650 10107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c: No bootstrap required, opened a new log
I20260926 04:03:46.677518 10107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } }
I20260926 04:03:46.677664 10107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:46.677695 10107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254d4aad57d545af94e69ea6a31d304c, State: Initialized, Role: FOLLOWER
I20260926 04:03:46.677767 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.672403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.0.200.190:42275)
0926 04:03:46.672608 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:46.672615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:46.672640 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:03:46.673086 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:46.673093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:46.673155 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:46.673272 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:46.673510 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.674477 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.675517 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.675968 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.676060 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.676805 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:46.676809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:46.676809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:46.676977 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:46.677012 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:46.677022 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:46.677078 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:46.677172 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 04:03:46.677233 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260926 04:03:46.677848 10107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } }
I20260926 04:03:46.677920 10107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:46.677945 10107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:46.677973 10107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:46.678670 10107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } }
I20260926 04:03:46.678791 10107 leader_election.cc:302] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 254d4aad57d545af94e69ea6a31d304c; no voters:
I20260926 04:03:46.678972 10107 leader_election.cc:288] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:46.679046 10112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:46.679232 10112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [term 1 LEADER]: Becoming Leader. State: Replica: 254d4aad57d545af94e69ea6a31d304c, State: Running, Role: LEADER
I20260926 04:03:46.679271 10107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:46.679322 10112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } }
I20260926 04:03:46.679786 10113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "254d4aad57d545af94e69ea6a31d304c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } } }
I20260926 04:03:46.679816 10114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 254d4aad57d545af94e69ea6a31d304c. Latest consensus state: current_term: 1 leader_uuid: "254d4aad57d545af94e69ea6a31d304c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254d4aad57d545af94e69ea6a31d304c" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 42275 } } }
I20260926 04:03:46.679919 10113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:46.679934 10114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:46.680167 10121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:46.680703 10121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:46.682229 10121 catalog_manager.cc:1347] Generated new cluster ID: 959028c14109422db20f39e50278590e
I20260926 04:03:46.682274 10121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:46.698128 10121 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:46.698876 10121 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:46.716750 10121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 254d4aad57d545af94e69ea6a31d304c: Generated new TSK 0
I20260926 04:03:46.716956 10121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:46.724141 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--builtin_ntp_servers=127.0.200.148:45593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 04:03:46.813810 10131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:46.814029 10131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:46.814051 10131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:46.815289 10131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:46.815330 10131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:46.815356 10131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:46.816709 10131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:46.816980 10131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:46.817307 10131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:46.822649 10139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:46.822707 10136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:46.822667 10137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:46.822865 10131 server_base.cc:1034] running on GCE node
I20260926 04:03:46.823130 10131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:46.823423 10131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:46.824600 10131 hybrid_clock.cc:648] HybridClock initialized: now 1790395426824590 us; error 61 us; skew 500 ppm
I20260926 04:03:46.825891 10131 webserver.cc:460] Webserver started at http://127.0.200.129:38637/ using document root <none> and password file <none>
I20260926 04:03:46.826148 10131 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:46.826221 10131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:46.826406 10131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:46.827600 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "41dda76b9d394b43915e99d0be29061a"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "1a57647e0c55708869974a8158199694"
server_key_iv: "a3f383ebe4533b6fd8a5ee1e533c9c22"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.827984 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "41dda76b9d394b43915e99d0be29061a"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "1a57647e0c55708869974a8158199694"
server_key_iv: "a3f383ebe4533b6fd8a5ee1e533c9c22"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.829430 10131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:46.830251 10145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.830590 10131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:46.830684 10131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "41dda76b9d394b43915e99d0be29061a"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "1a57647e0c55708869974a8158199694"
server_key_iv: "a3f383ebe4533b6fd8a5ee1e533c9c22"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.830781 10131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:46.851579 10131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:46.851912 10131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:46.852065 10131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:46.852308 10131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:46.852661 10131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:46.852720 10131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.852761 10131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:46.852788 10131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.858655 10131 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:37317
I20260926 04:03:46.858704 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:37317 every 8 connection(s)
I20260926 04:03:46.859069 10131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:46.859894 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10131
I20260926 04:03:46.859989 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:46.860210 802 external_mini_cluster.cc:1428] Setting key 307d4e54267f5aa243bd60ab7233bcbe
I20260926 04:03:46.861157 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--builtin_ntp_servers=127.0.200.148:45593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 04:03:46.865015 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.860389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:46157 (local address 127.0.200.190:42275)
0926 04:03:46.860436 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:46.860440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:46.860518 (+ 78us) server_negotiation.cc:407] Connection header received
0926 04:03:46.861175 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:46.861177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:46.861189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:46.861248 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:46.861964 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.862492 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.863526 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.863725 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.863815 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.864713 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:46.864715 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:46.864716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:46.864721 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:46.864738 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:46.864742 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:46.864795 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:46.864899 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 04:03:46.864931 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:46.865243 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.860128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42275 (local address 127.0.200.129:46157)
0926 04:03:46.860388 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:46.860438 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:46.861043 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:46.861333 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:46.861337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:46.861346 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:46.861853 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.861857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.862626 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.862633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.863428 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.863431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.863827 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.863828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.863965 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.864561 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:46.864561 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:46.864590 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:46.864613 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:46.864802 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:46.864807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:46.864810 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:46.864901 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 04:03:46.865005 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260926 04:03:46.865818 10259 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42275
I20260926 04:03:46.865912 10259 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:46.866115 10259 heartbeater.cc:511] Master 127.0.200.190:42275 requested a full tablet report, sending...
I20260926 04:03:46.866796 10071 ts_manager.cc:194] Registered new tserver with Master: 41dda76b9d394b43915e99d0be29061a (127.0.200.129:37317)
I20260926 04:03:46.867830 10071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.129:46157
W20260926 04:03:46.947494 10262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:46.947741 10262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:46.947767 10262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:46.949090 10262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:46.949134 10262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:46.949162 10262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:46.950951 10262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:46.951210 10262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:46.951512 10262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:46.957224 10267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:46.957376 10262 server_base.cc:1034] running on GCE node
W20260926 04:03:46.957435 10270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:46.957284 10268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:46.957731 10262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:46.958014 10262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:46.959180 10262 hybrid_clock.cc:648] HybridClock initialized: now 1790395426959169 us; error 49 us; skew 500 ppm
I20260926 04:03:46.960635 10262 webserver.cc:460] Webserver started at http://127.0.200.130:33121/ using document root <none> and password file <none>
I20260926 04:03:46.960920 10262 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:46.960994 10262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:46.961148 10262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:46.962329 10262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "414797a501b541f1b73def130361ee6c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "96ce4e99ac692244ffb69ca6b774fc5e"
server_key_iv: "369443e028e0da4d44895f6bec1e7eef"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.962764 10262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "414797a501b541f1b73def130361ee6c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "96ce4e99ac692244ffb69ca6b774fc5e"
server_key_iv: "369443e028e0da4d44895f6bec1e7eef"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.964183 10262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:46.964982 10276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.965169 10262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:46.965269 10262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "414797a501b541f1b73def130361ee6c"
format_stamp: "Formatted at 2026-09-26 04:03:46 on dist-test-slave-j8c1"
server_key: "96ce4e99ac692244ffb69ca6b774fc5e"
server_key_iv: "369443e028e0da4d44895f6bec1e7eef"
server_key_version: "encryptionkey@0"
I20260926 04:03:46.965355 10262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:46.980765 10262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:46.981137 10262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:46.981307 10262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:46.981591 10262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:46.981959 10262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:46.982018 10262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.982055 10262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:46.982103 10262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:46.989104 10262 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:37571
I20260926 04:03:46.989148 10389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:37571 every 8 connection(s)
I20260926 04:03:46.989511 10262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:46.995133 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.990329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:33409 (local address 127.0.200.190:42275)
0926 04:03:46.990405 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:46.990407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:46.990684 (+ 277us) server_negotiation.cc:407] Connection header received
0926 04:03:46.991268 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:46.991270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:46.991283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:46.991341 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:46.992031 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.992532 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.993579 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.993797 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.993889 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.994834 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:46.994836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:46.994836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:46.994841 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:46.994856 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:46.994861 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:46.994916 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:46.995015 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 04:03:46.995049 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 04:03:46.995417 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:46.990310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42275 (local address 127.0.200.130:33409)
0926 04:03:46.990519 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:46.990573 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:46.991109 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:46.991414 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:46.991417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:46.991426 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:46.991920 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.991924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.992632 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.992633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.993476 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:46.993479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:46.993900 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:46.993901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:46.994034 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:46.994682 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:46.994683 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:46.994713 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:46.994738 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:46.994919 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:46.994923 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:46.994927 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:46.995017 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 04:03:46.995124 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260926 04:03:46.996038 10390 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42275
I20260926 04:03:46.996124 10390 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:46.996395 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10262
I20260926 04:03:46.996419 10390 heartbeater.cc:511] Master 127.0.200.190:42275 requested a full tablet report, sending...
I20260926 04:03:46.996503 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:46.996809 802 external_mini_cluster.cc:1428] Setting key bce464b38643086ed59cb68c9d5ed674
I20260926 04:03:46.996914 10071 ts_manager.cc:194] Registered new tserver with Master: 414797a501b541f1b73def130361ee6c (127.0.200.130:37571)
I20260926 04:03:46.997392 10071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.130:33409
I20260926 04:03:46.998240 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--builtin_ntp_servers=127.0.200.148:45593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 04:03:47.080351 10393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:47.080644 10393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:47.080680 10393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:47.081909 10393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:47.081986 10393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:47.082028 10393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:47.083545 10393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:42275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:47.083830 10393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:47.084149 10393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:47.089685 10401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.089694 10399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.089710 10398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.089952 10393 server_base.cc:1034] running on GCE node
I20260926 04:03:47.090199 10393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:47.090477 10393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:47.091691 10393 hybrid_clock.cc:648] HybridClock initialized: now 1790395427091672 us; error 60 us; skew 500 ppm
I20260926 04:03:47.092973 10393 webserver.cc:460] Webserver started at http://127.0.200.131:44681/ using document root <none> and password file <none>
I20260926 04:03:47.093221 10393 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:47.093330 10393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:47.093497 10393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:47.094693 10393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "c62fed4b670f4b17944509f6a46d3a8b"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "24299fd07323c2a401389166cd636d37"
server_key_iv: "96f580cc01897fb9ffc9e734da3eff65"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.095074 10393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "c62fed4b670f4b17944509f6a46d3a8b"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "24299fd07323c2a401389166cd636d37"
server_key_iv: "96f580cc01897fb9ffc9e734da3eff65"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.096483 10393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:47.097277 10407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.097463 10393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:47.097561 10393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "c62fed4b670f4b17944509f6a46d3a8b"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "24299fd07323c2a401389166cd636d37"
server_key_iv: "96f580cc01897fb9ffc9e734da3eff65"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.097653 10393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:47.110163 10393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:47.110524 10393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:47.110733 10393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:47.110989 10393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:47.111358 10393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:47.111415 10393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.111450 10393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:47.111477 10393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.117559 10393 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:37961
I20260926 04:03:47.117614 10520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:37961 every 8 connection(s)
I20260926 04:03:47.117973 10393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:47.123354 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10393
I20260926 04:03:47.123476 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:47.123641 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.118825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:35913 (local address 127.0.200.190:42275)
0926 04:03:47.118895 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.118897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.119143 (+ 246us) server_negotiation.cc:407] Connection header received
0926 04:03:47.119668 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.119669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.119679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.119732 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.120428 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:47.120946 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:47.121960 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:47.122155 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:47.122226 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:47.123363 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:47.123364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:47.123364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:47.123369 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:47.123382 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:47.123387 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:47.123435 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:47.123556 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 04:03:47.123579 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:03:47.123667 802 external_mini_cluster.cc:1428] Setting key 0e03b5fa5909e88e2b12bb4ce749471d
I20260926 04:03:47.124035 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.118841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:42275 (local address 127.0.200.131:35913)
0926 04:03:47.119013 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:47.119047 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:47.119577 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:47.119810 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:47.119814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:47.119822 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:47.120309 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:47.120313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:47.121056 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:47.121057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:47.121859 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:47.121862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:47.122324 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:47.122325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:47.122447 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:47.123167 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:47.123167 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:47.123211 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:47.123252 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:47.123460 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:47.123467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:47.123471 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:47.123549 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 04:03:47.123670 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260926 04:03:47.124454 10521 heartbeater.cc:347] Connected to a master server at 127.0.200.190:42275
I20260926 04:03:47.124562 10521 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:47.124758 10521 heartbeater.cc:511] Master 127.0.200.190:42275 requested a full tablet report, sending...
I20260926 04:03:47.125114 10071 ts_manager.cc:194] Registered new tserver with Master: c62fed4b670f4b17944509f6a46d3a8b (127.0.200.131:37961)
I20260926 04:03:47.125504 10071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.131:35913
I20260926 04:03:47.135545 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 04:03:47.137451 10110 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:47.136927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.0.200.190:42275)
0926 04:03:47.137006 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.137009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.137094 (+ 85us) server_negotiation.cc:407] Connection header received
0926 04:03:47.137162 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.137163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.137174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.137224 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.137365 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53138: BlockingRecv error: recv got EOF from 127.0.0.1:53138 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 04:03:47.137555 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10131
I20260926 04:03:47.142216 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10262
I20260926 04:03:47.146531 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10393
I20260926 04:03:47.151252 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10040
2026-09-26T04:03:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (631 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T04:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:47Z Disabled control of system clock
I20260926 04:03:47.178237 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45151
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 04:03:47.263245 10532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:47.263464 10532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:47.263485 10532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:47.264621 10532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:47.264663 10532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:47.264678 10532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:47.264690 10532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:47.264703 10532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:47.266006 10532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:45151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:47.266212 10532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:47.266456 10532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:47.271400 10538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.271396 10540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.271431 10532 server_base.cc:1034] running on GCE node
W20260926 04:03:47.271430 10537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.271878 10532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:47.272200 10532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:47.273470 10532 hybrid_clock.cc:648] HybridClock initialized: now 1790395427273446 us; error 61 us; skew 500 ppm
I20260926 04:03:47.274930 10532 webserver.cc:460] Webserver started at http://127.0.200.190:34443/ using document root <none> and password file <none>
I20260926 04:03:47.275194 10532 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:47.275269 10532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:47.275429 10532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:47.276543 10532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "f7f15c369a0e4708a00dffae977a0dbe"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "729c31215c50c828019e60089b7e195b"
server_key_iv: "8c22f1c1e0c002e80e354c9cc8081700"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.276923 10532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "f7f15c369a0e4708a00dffae977a0dbe"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "729c31215c50c828019e60089b7e195b"
server_key_iv: "8c22f1c1e0c002e80e354c9cc8081700"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.278329 10532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:47.279284 10546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.279484 10532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:47.279592 10532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "f7f15c369a0e4708a00dffae977a0dbe"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "729c31215c50c828019e60089b7e195b"
server_key_iv: "8c22f1c1e0c002e80e354c9cc8081700"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.279690 10532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:47.283116 10532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:47.283412 10532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:47.283560 10532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:47.287490 10532 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:45151
I20260926 04:03:47.287524 10598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:45151 every 8 connection(s)
I20260926 04:03:47.287865 10532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:47.288532 10599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:47.291328 10599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe: Bootstrap starting.
I20260926 04:03:47.291894 10599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:47.292129 10599 log.cc:824] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:47.292886 10599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe: No bootstrap required, opened a new log
I20260926 04:03:47.292937 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10532
I20260926 04:03:47.293017 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:47.293217 802 external_mini_cluster.cc:1428] Setting key 58b61b0b767ae2022bb44a22b1543371
I20260926 04:03:47.294909 10599 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } }
I20260926 04:03:47.295074 10599 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:47.295112 10599 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f15c369a0e4708a00dffae977a0dbe, State: Initialized, Role: FOLLOWER
I20260926 04:03:47.295288 10599 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } }
I20260926 04:03:47.295403 10599 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:47.295460 10599 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:47.295516 10599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:47.296218 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.293777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.0.200.190:45151)
0926 04:03:47.293981 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.293987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.294013 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:03:47.294890 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.294896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.294934 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.295054 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.295755 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:47.295759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:47.295759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:47.295763 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:47.295816 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:47.295830 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:47.295903 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:47.295954 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 04:03:47.296011 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260926 04:03:47.296273 10599 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } }
I20260926 04:03:47.296361 10599 leader_election.cc:302] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7f15c369a0e4708a00dffae977a0dbe; no voters:
I20260926 04:03:47.296509 10599 leader_election.cc:288] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:47.296782 10604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:47.296825 10599 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:47.296926 10604 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [term 1 LEADER]: Becoming Leader. State: Replica: f7f15c369a0e4708a00dffae977a0dbe, State: Running, Role: LEADER
I20260926 04:03:47.297070 10604 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } }
I20260926 04:03:47.297429 10606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7f15c369a0e4708a00dffae977a0dbe. Latest consensus state: current_term: 1 leader_uuid: "f7f15c369a0e4708a00dffae977a0dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } } }
I20260926 04:03:47.297529 10606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:47.297441 10605 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7f15c369a0e4708a00dffae977a0dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7f15c369a0e4708a00dffae977a0dbe" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 45151 } } }
I20260926 04:03:47.297712 10605 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:47.297717 10613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:47.298288 10613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:47.299883 10613 catalog_manager.cc:1347] Generated new cluster ID: b01a309c77b846aaabe607f942f3473d
I20260926 04:03:47.299942 10613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:03:47.313297 10613 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:47.313531 10613 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:47.322096 10613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7f15c369a0e4708a00dffae977a0dbe: Generated new TSK 0
I20260926 04:03:47.322383 10613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:03:47.325069 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--builtin_ntp_servers=127.0.200.148:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 04:03:47.412741 10623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:47.412976 10623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:47.413009 10623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:47.414237 10623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:47.414311 10623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:47.414355 10623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:47.415797 10623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:47.416065 10623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:47.416373 10623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:47.421799 10628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.421841 10631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.421793 10629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.422189 10623 server_base.cc:1034] running on GCE node
I20260926 04:03:47.422408 10623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:47.422675 10623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:47.423848 10623 hybrid_clock.cc:648] HybridClock initialized: now 1790395427423838 us; error 34 us; skew 500 ppm
I20260926 04:03:47.425130 10623 webserver.cc:460] Webserver started at http://127.0.200.129:42945/ using document root <none> and password file <none>
I20260926 04:03:47.425418 10623 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:47.425489 10623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:47.425633 10623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:47.426843 10623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "bffacd93161b49599ccab8ef612d94cb"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "630ada59aa38baba25b6bf5b12068806"
server_key_iv: "88dc065b51d4d25b15574f94b26083a1"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.427202 10623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "bffacd93161b49599ccab8ef612d94cb"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "630ada59aa38baba25b6bf5b12068806"
server_key_iv: "88dc065b51d4d25b15574f94b26083a1"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.428633 10623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:47.429396 10637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.429582 10623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:03:47.429672 10623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "bffacd93161b49599ccab8ef612d94cb"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "630ada59aa38baba25b6bf5b12068806"
server_key_iv: "88dc065b51d4d25b15574f94b26083a1"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.429733 10623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:47.433115 10623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:47.433410 10623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:47.433521 10623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:47.433708 10623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:47.433961 10623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:47.433991 10623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.434011 10623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:47.434024 10623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.439621 10623 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:39465
I20260926 04:03:47.439697 10750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:39465 every 8 connection(s)
I20260926 04:03:47.440049 10623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:03:47.443380 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.440840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:54775 (local address 127.0.200.190:45151)
0926 04:03:47.440889 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.440891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.441156 (+ 265us) server_negotiation.cc:407] Connection header received
0926 04:03:47.441696 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.441698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.441704 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.441763 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.443079 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:47.443081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:47.443081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:47.443087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:47.443105 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:47.443110 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:47.443187 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:47.443282 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 04:03:47.443298 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 04:03:47.443624 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.440835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45151 (local address 127.0.200.129:54775)
0926 04:03:47.441029 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:47.441062 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:47.441571 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:47.441847 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:47.441851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:47.441859 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:47.442915 (+ 1056us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:47.442915 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:47.442945 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:47.442973 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:47.443185 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:47.443192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:47.443195 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:47.443293 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 04:03:47.443358 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260926 04:03:47.443691 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10623
I20260926 04:03:47.443810 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:47.443985 802 external_mini_cluster.cc:1428] Setting key 4920f073801290900f9c9571382ca22c
I20260926 04:03:47.444543 10751 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45151
I20260926 04:03:47.444650 10751 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:47.444834 10751 heartbeater.cc:511] Master 127.0.200.190:45151 requested a full tablet report, sending...
I20260926 04:03:47.445149 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--builtin_ntp_servers=127.0.200.148:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 04:03:47.445420 10562 ts_manager.cc:194] Registered new tserver with Master: bffacd93161b49599ccab8ef612d94cb (127.0.200.129:39465)
W20260926 04:03:47.527801 10754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:47.528012 10754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:47.528038 10754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:47.529249 10754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:47.529294 10754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:47.529320 10754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:47.530781 10754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:47.531033 10754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:47.531337 10754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:47.536484 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.536491 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.536535 10754 server_base.cc:1034] running on GCE node
W20260926 04:03:47.536484 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.536978 10754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:47.537191 10754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:47.538429 10754 hybrid_clock.cc:648] HybridClock initialized: now 1790395427538407 us; error 36 us; skew 500 ppm
I20260926 04:03:47.539934 10754 webserver.cc:460] Webserver started at http://127.0.200.130:36257/ using document root <none> and password file <none>
I20260926 04:03:47.540210 10754 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:47.540297 10754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:47.540477 10754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:47.541594 10754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "7d4bf488b9124a83aaf3fbd5a2d0b606"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "8478a1691ca0720392cd1079bb9574d2"
server_key_iv: "be82dd47801d5152cd8673255b08e29b"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.541965 10754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "7d4bf488b9124a83aaf3fbd5a2d0b606"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "8478a1691ca0720392cd1079bb9574d2"
server_key_iv: "be82dd47801d5152cd8673255b08e29b"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.543449 10754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:47.544310 10768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.544519 10754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:47.544627 10754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "7d4bf488b9124a83aaf3fbd5a2d0b606"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "8478a1691ca0720392cd1079bb9574d2"
server_key_iv: "be82dd47801d5152cd8673255b08e29b"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.544715 10754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:47.548322 10754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:47.548601 10754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:47.548760 10754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:47.548991 10754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:47.549322 10754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:47.549381 10754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.549415 10754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:47.549441 10754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.555482 10754 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:40981
I20260926 04:03:47.555518 10881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:40981 every 8 connection(s)
I20260926 04:03:47.555850 10754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:47.559362 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.556631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:50981 (local address 127.0.200.190:45151)
0926 04:03:47.556701 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.556703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.556948 (+ 245us) server_negotiation.cc:407] Connection header received
0926 04:03:47.557498 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.557500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.557509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.557610 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.559025 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:47.559026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:47.559027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:47.559033 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:47.559059 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:47.559064 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:47.559130 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:47.559288 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 04:03:47.559305 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 04:03:47.559459 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.556604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45151 (local address 127.0.200.130:50981)
0926 04:03:47.556829 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:47.556863 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:47.557328 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:47.557728 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:47.557731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:47.557741 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:47.558825 (+ 1084us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:47.558826 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:47.558857 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:47.558888 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:47.559139 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:47.559143 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:47.559145 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:03:47.559206 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 04:03:47.559261 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260926 04:03:47.560186 10882 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45151
I20260926 04:03:47.560307 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10754
I20260926 04:03:47.560375 10882 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:47.560384 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:47.560544 10882 heartbeater.cc:511] Master 127.0.200.190:45151 requested a full tablet report, sending...
I20260926 04:03:47.560577 802 external_mini_cluster.cc:1428] Setting key ae528b43368a5829b8e73a5391bf5ef8
I20260926 04:03:47.561015 10563 ts_manager.cc:194] Registered new tserver with Master: 7d4bf488b9124a83aaf3fbd5a2d0b606 (127.0.200.130:40981)
I20260926 04:03:47.561610 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--builtin_ntp_servers=127.0.200.148:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 04:03:47.646066 10885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:47.646299 10885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:47.646323 10885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:47.647670 10885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:47.647722 10885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:47.647750 10885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:47.649204 10885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:47.649427 10885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:47.649690 10885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:47.654830 10893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:47.654853 10891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.655050 10885 server_base.cc:1034] running on GCE node
W20260926 04:03:47.655090 10890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:47.655398 10885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:47.655690 10885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:47.656873 10885 hybrid_clock.cc:648] HybridClock initialized: now 1790395427656852 us; error 35 us; skew 500 ppm
I20260926 04:03:47.658367 10885 webserver.cc:460] Webserver started at http://127.0.200.131:43097/ using document root <none> and password file <none>
I20260926 04:03:47.658699 10885 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:47.658780 10885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:47.658941 10885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:47.660054 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "31790a90bf7f47dc943cdbe2b2c5d260"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "cc38f8040662f394f830015f0ceb74ff"
server_key_iv: "ac7e46b696c5f5a27f901ef3d5ab0604"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.660436 10885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "31790a90bf7f47dc943cdbe2b2c5d260"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "cc38f8040662f394f830015f0ceb74ff"
server_key_iv: "ac7e46b696c5f5a27f901ef3d5ab0604"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.661787 10885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:47.662747 10899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.662926 10885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:47.663036 10885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "31790a90bf7f47dc943cdbe2b2c5d260"
format_stamp: "Formatted at 2026-09-26 04:03:47 on dist-test-slave-j8c1"
server_key: "cc38f8040662f394f830015f0ceb74ff"
server_key_iv: "ac7e46b696c5f5a27f901ef3d5ab0604"
server_key_version: "encryptionkey@0"
I20260926 04:03:47.663115 10885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:47.666610 10885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:47.666911 10885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:47.667080 10885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:47.667295 10885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:47.667605 10885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:47.667661 10885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.667701 10885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:47.667728 10885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:47.673898 10885 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:42625
I20260926 04:03:47.673934 11012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:42625 every 8 connection(s)
I20260926 04:03:47.674324 10885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:47.677155 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 10885
I20260926 04:03:47.677281 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:47.677469 802 external_mini_cluster.cc:1428] Setting key e612d22e2c48d9bed21a2b7526c15ed5
I20260926 04:03:47.678363 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.675465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:58353 (local address 127.0.200.190:45151)
0926 04:03:47.675542 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.675544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.675783 (+ 239us) server_negotiation.cc:407] Connection header received
0926 04:03:47.676361 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.676362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.676371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.676433 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.678053 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:47.678054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:47.678055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:03:47.678060 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:47.678076 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:03:47.678081 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:47.678160 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:47.678272 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 04:03:47.678291 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:03:47.678615 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:47.675408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:45151 (local address 127.0.200.131:58353)
0926 04:03:47.675658 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:47.675694 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:47.676216 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:47.676514 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:47.676518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:47.676528 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:47.677898 (+ 1370us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:03:47.677899 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:47.677930 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:03:47.677959 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:47.678178 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:47.678183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:47.678186 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:03:47.678282 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 04:03:47.678355 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260926 04:03:47.679021 11013 heartbeater.cc:347] Connected to a master server at 127.0.200.190:45151
I20260926 04:03:47.679136 11013 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:47.679293 11013 heartbeater.cc:511] Master 127.0.200.190:45151 requested a full tablet report, sending...
I20260926 04:03:47.679695 10563 ts_manager.cc:194] Registered new tserver with Master: 31790a90bf7f47dc943cdbe2b2c5d260 (127.0.200.131:42625)
I20260926 04:03:47.689849 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 04:03:47.692037 10603 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:03:47.691455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.0.200.190:45151)
0926 04:03:47.691539 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:47.691541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:47.691636 (+ 95us) server_negotiation.cc:407] Connection header received
0926 04:03:47.691719 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:47.691720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:47.691726 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:47.691800 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:47.691960 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38586: BlockingRecv error: recv got EOF from 127.0.0.1:38586 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 04:03:47.692087 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10623
I20260926 04:03:47.697266 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10754
I20260926 04:03:47.701550 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10885
I20260926 04:03:47.706149 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 10532
2026-09-26T04:03:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (555 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:47 dist-test-slave-j8c1 krb5kdc[11023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:47 dist-test-slave-j8c1 krb5kdc[11023](info): set up 2 sockets
Sep 26 04:03:47 dist-test-slave-j8c1 krb5kdc[11023](info): commencing operation
krb5kdc: starting...
W20260926 04:03:48.743778 802 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:48 dist-test-slave-j8c1 krb5kdc[11023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:03:48.869792 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36881
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:48.951131 11036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:48.951367 11036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:48.951393 11036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:48.952587 11036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:03:48.952637 11036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:48.952652 11036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:48.952665 11036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:03:48.952678 11036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:03:48.954167 11036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:48.954385 11036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:48.954596 11036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:48.959846 11044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:48.959818 11042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:48.959950 11041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:48.960134 11036 server_base.cc:1034] running on GCE node
I20260926 04:03:48.960374 11036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:48.960673 11036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:48.961882 11036 hybrid_clock.cc:648] HybridClock initialized: now 1790395428961858 us; error 46 us; skew 500 ppm
Sep 26 04:03:48 dist-test-slave-j8c1 krb5kdc[11023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395428, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:48.964767 11036 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:03:48.965281 11036 webserver.cc:460] Webserver started at http://127.0.200.190:44563/ using document root <none> and password file <none>
I20260926 04:03:48.965519 11036 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:48.965590 11036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:48.965745 11036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:48.966913 11036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "703b54a6784b491c9d4a96deacb46f77"
format_stamp: "Formatted at 2026-09-26 04:03:48 on dist-test-slave-j8c1"
server_key: "35f88ef465dac419623b0c5b8cd56015"
server_key_iv: "ae6011f774dbfef875b43bcbecf1598e"
server_key_version: "encryptionkey@0"
I20260926 04:03:48.967291 11036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "703b54a6784b491c9d4a96deacb46f77"
format_stamp: "Formatted at 2026-09-26 04:03:48 on dist-test-slave-j8c1"
server_key: "35f88ef465dac419623b0c5b8cd56015"
server_key_iv: "ae6011f774dbfef875b43bcbecf1598e"
server_key_version: "encryptionkey@0"
I20260926 04:03:48.968740 11036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:03:48.969763 11051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:48.970062 11036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:48.970189 11036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "703b54a6784b491c9d4a96deacb46f77"
format_stamp: "Formatted at 2026-09-26 04:03:48 on dist-test-slave-j8c1"
server_key: "35f88ef465dac419623b0c5b8cd56015"
server_key_iv: "ae6011f774dbfef875b43bcbecf1598e"
server_key_version: "encryptionkey@0"
I20260926 04:03:48.970293 11036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:48.982045 11036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:48.984575 11036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:48.984764 11036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:48.989074 11036 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:36881
I20260926 04:03:48.989108 11103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:36881 every 8 connection(s)
I20260926 04:03:48.989483 11036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:03:48.990658 11104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:48.992976 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: Bootstrap starting.
I20260926 04:03:48.993543 11104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:48.993774 11104 log.cc:824] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:48.994515 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: No bootstrap required, opened a new log
I20260926 04:03:48.994777 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 11036
I20260926 04:03:48.994877 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:03:48.995078 802 external_mini_cluster.cc:1428] Setting key 1fd2a4de4ff0ee3348112671a6ff4a3f
I20260926 04:03:48.996793 11104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } }
I20260926 04:03:48.996982 11104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:48.997023 11104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703b54a6784b491c9d4a96deacb46f77, State: Initialized, Role: FOLLOWER
I20260926 04:03:48.997188 11104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } }
I20260926 04:03:48.997285 11104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:03:48.997319 11104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:03:48.997376 11104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:48.998476 11104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } }
I20260926 04:03:48.998631 11104 leader_election.cc:302] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 703b54a6784b491c9d4a96deacb46f77; no voters:
I20260926 04:03:48.998829 11104 leader_election.cc:288] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:03:48.998852 11109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:48.998958 11109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [term 1 LEADER]: Becoming Leader. State: Replica: 703b54a6784b491c9d4a96deacb46f77, State: Running, Role: LEADER
I20260926 04:03:48.999131 11104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:03:48.999117 11109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } }
I20260926 04:03:48.999482 11111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "703b54a6784b491c9d4a96deacb46f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } } }
I20260926 04:03:48.999575 11111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:48.999466 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 703b54a6784b491c9d4a96deacb46f77. Latest consensus state: current_term: 1 leader_uuid: "703b54a6784b491c9d4a96deacb46f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "703b54a6784b491c9d4a96deacb46f77" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36881 } } }
I20260926 04:03:48.999912 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77 [sys.catalog]: This master's current role is: LEADER
I20260926 04:03:49.000138 11115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:03:49.000746 11115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:03:49.002491 11115 catalog_manager.cc:1347] Generated new cluster ID: 0feaf7109c7b417a8d4f8149bba18a04
I20260926 04:03:49.002566 11115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:49.007048 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:48.995747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.0.200.190:36881)
0926 04:03:48.995942 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:48.995951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:48.995978 (+ 27us) server_negotiation.cc:407] Connection header received
0926 04:03:48.996773 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:48.996781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:48.996836 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:48.997013 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:48.997801 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:48.998890 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:48.999998 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.000657 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.000762 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.003531 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:49.003535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:49.003536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:49.003548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:49.005321 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.005712 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.005715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.005716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.005830 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.006042 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.006042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.006043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.006291 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:49.006467 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.006597 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.006740 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260926 04:03:49.022703 11115 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:03:49.023324 11115 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:03:49.025781 11115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: Generated new TSK 0
I20260926 04:03:49.025988 11115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:03:49.060169 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:49.153970 11130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:49.154201 11130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:49.154228 11130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:49.155603 11130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:49.155659 11130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:49.155689 11130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:03:49.157261 11130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:49.157542 11130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:49.157805 11130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:49.163107 11135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.163121 11136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.163322 11138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:49.163343 11130 server_base.cc:1034] running on GCE node
I20260926 04:03:49.163539 11130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:49.163751 11130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:49.164911 11130 hybrid_clock.cc:648] HybridClock initialized: now 1790395429164891 us; error 38 us; skew 500 ppm
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:49.167711 11130 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:03:49.168265 11130 webserver.cc:460] Webserver started at http://127.0.200.129:41131/ using document root <none> and password file <none>
I20260926 04:03:49.168517 11130 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:49.168576 11130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:49.168761 11130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:49.169874 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "6126d691769b42a48700791e6b110e91"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "9a7da0b13a16559922a8e535d42d967f"
server_key_iv: "a1f369fb4867fbed73b3216e8bafb973"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.170246 11130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "6126d691769b42a48700791e6b110e91"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "9a7da0b13a16559922a8e535d42d967f"
server_key_iv: "a1f369fb4867fbed73b3216e8bafb973"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.171711 11130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:03:49.172575 11145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.172919 11130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:49.173061 11130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "6126d691769b42a48700791e6b110e91"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "9a7da0b13a16559922a8e535d42d967f"
server_key_iv: "a1f369fb4867fbed73b3216e8bafb973"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.173148 11130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:49.187737 11130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:49.190136 11130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:49.190303 11130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:49.190609 11130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:49.190951 11130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:49.191006 11130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.191048 11130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:49.191076 11130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.199034 11130 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:37435
I20260926 04:03:49.199111 11258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:37435 every 8 connection(s)
I20260926 04:03:49.199506 11130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:49.206507 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 11130
I20260926 04:03:49.206646 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:03:49.206825 802 external_mini_cluster.cc:1428] Setting key b0578a9b103c7fb30882cf1ffe07bc55
I20260926 04:03:49.211166 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.200309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:37731 (local address 127.0.200.190:36881)
0926 04:03:49.200357 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.200359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.200621 (+ 262us) server_negotiation.cc:407] Connection header received
0926 04:03:49.201245 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.201247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.201260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.201330 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:49.202687 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.203408 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.204579 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.204776 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.204836 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.207107 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:49.207109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:49.207110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:49.207117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:49.209680 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.210102 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.210104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.210104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.210172 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.210437 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.210438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.210439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.210629 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:49.210695 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.210981 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.211075 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:03:49.211354 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.200321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36881 (local address 127.0.200.129:37731)
0926 04:03:49.200495 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.200531 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.201126 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.201408 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.201413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.202247 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:49.202581 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.202588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.203499 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.203500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.204501 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.204504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.204853 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.204853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.204949 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.205490 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:49.205498 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:49.207000 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:49.209771 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.209779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.209782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.210022 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.210239 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.210240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.210240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.210369 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.210685 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:49.210691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:49.210840 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.210953 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.211112 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260926 04:03:49.211835 11259 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36881
I20260926 04:03:49.211938 11259 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:49.212090 11259 heartbeater.cc:511] Master 127.0.200.190:36881 requested a full tablet report, sending...
I20260926 04:03:49.212604 11068 ts_manager.cc:194] Registered new tserver with Master: 6126d691769b42a48700791e6b110e91 (127.0.200.129:37435)
I20260926 04:03:49.213164 11068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:37731
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:03:49.233544 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:49.315356 11264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:49.315639 11264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:49.315676 11264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:49.316876 11264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:49.316948 11264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:49.316994 11264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:03:49.318487 11264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:49.318789 11264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:49.319036 11264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:49.324165 11270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.324194 11272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.324332 11269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:49.324385 11264 server_base.cc:1034] running on GCE node
I20260926 04:03:49.324606 11264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:49.324900 11264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:49.326043 11264 hybrid_clock.cc:648] HybridClock initialized: now 1790395429326017 us; error 48 us; skew 500 ppm
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:49.329010 11264 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:03:49.329465 11264 webserver.cc:460] Webserver started at http://127.0.200.130:43019/ using document root <none> and password file <none>
I20260926 04:03:49.329710 11264 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:49.329787 11264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:49.329941 11264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:49.331102 11264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "0ac2e9d22f33424182530ec239024816"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "7e1ea3832071c85ba257c60ba25be057"
server_key_iv: "4bec73413c9c43c65392bb06247fe701"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.331485 11264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "0ac2e9d22f33424182530ec239024816"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "7e1ea3832071c85ba257c60ba25be057"
server_key_iv: "4bec73413c9c43c65392bb06247fe701"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.332834 11264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 04:03:49.333618 11279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.333801 11264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:03:49.333906 11264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "0ac2e9d22f33424182530ec239024816"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "7e1ea3832071c85ba257c60ba25be057"
server_key_iv: "4bec73413c9c43c65392bb06247fe701"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.333981 11264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:49.356323 11264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:49.358798 11264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:49.358978 11264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:49.359221 11264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:49.359542 11264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:49.359597 11264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.359633 11264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:49.359661 11264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.365499 11264 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:39947
I20260926 04:03:49.365537 11392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:39947 every 8 connection(s)
I20260926 04:03:49.365880 11264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:03:49.368994 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 11264
I20260926 04:03:49.369091 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:03:49.369268 802 external_mini_cluster.cc:1428] Setting key 543489a90a5be271887dec218871ca7d
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:49.377219 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.366664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:51229 (local address 127.0.200.190:36881)
0926 04:03:49.366712 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.366715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.366911 (+ 196us) server_negotiation.cc:407] Connection header received
0926 04:03:49.367478 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.367480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.367493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.367554 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:49.368547 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.369249 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.370488 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.370831 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.370908 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.373783 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:49.373785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:49.373786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:49.373794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:49.375400 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.375841 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.375843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.375843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.375925 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.376232 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.376233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.376234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.376360 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:49.376441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.376731 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.377101 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:49.377388 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.366627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36881 (local address 127.0.200.130:51229)
0926 04:03:49.366776 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.366822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.367359 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.367623 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.367627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.368225 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:49.368418 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.368422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.369367 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.369368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.370392 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.370395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.370915 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.370916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.371018 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.371647 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:49.371659 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:49.373672 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:49.375492 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.375501 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.375503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.375745 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.375982 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.375982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.375983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.376164 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.376432 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:49.376438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:49.376595 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.376712 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.377117 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:03:49.377940 11393 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36881
I20260926 04:03:49.378036 11393 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:49.378232 11393 heartbeater.cc:511] Master 127.0.200.190:36881 requested a full tablet report, sending...
I20260926 04:03:49.378679 11068 ts_manager.cc:194] Registered new tserver with Master: 0ac2e9d22f33424182530ec239024816 (127.0.200.130:39947)
I20260926 04:03:49.379145 11068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:51229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:03:49.398669 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:03:49.481072 11398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:03:49.481303 11398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:03:49.481329 11398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:03:49.482810 11398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:03:49.482877 11398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:03:49.482908 11398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:03:49.484659 11398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:03:49.484947 11398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:03:49.485204 11398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:03:49.490792 11403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.490756 11406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:03:49.490794 11404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:03:49.491132 11398 server_base.cc:1034] running on GCE node
I20260926 04:03:49.491334 11398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:03:49.491580 11398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:03:49.492743 11398 hybrid_clock.cc:648] HybridClock initialized: now 1790395429492727 us; error 29 us; skew 500 ppm
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:03:49.495666 11398 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:03:49.496178 11398 webserver.cc:460] Webserver started at http://127.0.200.131:44033/ using document root <none> and password file <none>
I20260926 04:03:49.496368 11398 fs_manager.cc:359] Metadata directory not provided
I20260926 04:03:49.496408 11398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:03:49.496508 11398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:03:49.497612 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "db53b0ef92cb46fda557280c8f6b5982"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "e15da0205e4d1e90a29becb62455233a"
server_key_iv: "65eb77925b75180dd34bba30d49de903"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.497915 11398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "db53b0ef92cb46fda557280c8f6b5982"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "e15da0205e4d1e90a29becb62455233a"
server_key_iv: "65eb77925b75180dd34bba30d49de903"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.499440 11398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:03:49.500262 11413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.500459 11398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:03:49.500604 11398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "db53b0ef92cb46fda557280c8f6b5982"
format_stamp: "Formatted at 2026-09-26 04:03:49 on dist-test-slave-j8c1"
server_key: "e15da0205e4d1e90a29becb62455233a"
server_key_iv: "65eb77925b75180dd34bba30d49de903"
server_key_version: "encryptionkey@0"
I20260926 04:03:49.500695 11398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:03:49.512668 11398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:03:49.515149 11398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:03:49.515321 11398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:03:49.515560 11398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:03:49.515873 11398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:03:49.515926 11398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.515966 11398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:03:49.515993 11398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:03:49.521971 11398 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:36641
I20260926 04:03:49.522050 11526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:36641 every 8 connection(s)
I20260926 04:03:49.522351 11398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:03:49.523972 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 11398
I20260926 04:03:49.524071 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:03:49.524233 802 external_mini_cluster.cc:1428] Setting key cb778a0a746734ba88b1c69c0e7f0910
Sep 26 04:03:49 dist-test-slave-j8c1 krb5kdc[11023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395429, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:03:49.532119 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.523143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:41953 (local address 127.0.200.190:36881)
0926 04:03:49.523192 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.523195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.523474 (+ 279us) server_negotiation.cc:407] Connection header received
0926 04:03:49.523935 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.523936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.523949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.524023 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:49.525330 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.525948 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.527011 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.527202 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.527259 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.529465 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:49.529467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:49.529467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:49.529474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:49.530847 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.531252 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.531253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.531253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.531304 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.531596 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.531596 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.531597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.531700 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:49.531752 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.531990 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.532051 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:03:49.532438 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.523128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36881 (local address 127.0.200.131:41953)
0926 04:03:49.523305 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.523342 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.523840 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.524150 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.524155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.524952 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:03:49.525239 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.525244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.526014 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.526015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.526931 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.526933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.527277 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.527278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.527381 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.527996 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:03:49.528004 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:03:49.529380 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:03:49.530919 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.530924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.530926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.531156 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.531391 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:03:49.531392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:03:49.531392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:03:49.531514 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:03:49.531758 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:03:49.531762 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:03:49.531876 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.531958 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.532085 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260926 04:03:49.533000 11527 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36881
I20260926 04:03:49.533118 11527 heartbeater.cc:464] Registering TS with master...
I20260926 04:03:49.533300 11527 heartbeater.cc:511] Master 127.0.200.190:36881 requested a full tablet report, sending...
I20260926 04:03:49.533788 11068 ts_manager.cc:194] Registered new tserver with Master: db53b0ef92cb46fda557280c8f6b5982 (127.0.200.131:36641)
I20260926 04:03:49.534174 11068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:41953
I20260926 04:03:49.536010 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:03:49.544296 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.537364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.0.200.190:36881)
0926 04:03:49.537439 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.537441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.537550 (+ 109us) server_negotiation.cc:407] Connection header received
0926 04:03:49.537626 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.537627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.537637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.537695 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:03:49.538323 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.538888 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.539615 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.539810 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.539902 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.541514 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:03:49.541516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:03:49.541517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:03:49.541523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:03:49.543025 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.543412 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.543414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.543414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.543494 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:03:49.543719 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:03:49.543720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:03:49.543720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:03:49.543879 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:03:49.543992 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.544104 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.544188 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 04:03:49.546155 11068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:03:49.548095 11068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:03:49.555038 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.550829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:36641 (local address 127.0.0.1:52866)
0926 04:03:49.551002 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.551036 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.551096 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.551362 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.551365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.551367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:49.551445 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.551448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.552502 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.552503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.553631 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.553634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.554689 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.554690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.554780 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.554796 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.554890 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.554950 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260926 04:03:49.555128 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.550960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.0.200.131:36641)
0926 04:03:49.551201 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.551208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.551223 (+ 15us) server_negotiation.cc:407] Connection header received
0926 04:03:49.551253 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.551254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.551267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.551358 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:49.551536 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.552392 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.553732 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.554598 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.554669 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.554730 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.554901 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.554943 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":129,"threads_started":1}
I20260926 04:03:49.555672 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.550775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:39947 (local address 127.0.0.1:37842)
0926 04:03:49.550977 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.551021 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.551096 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.551652 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.551652 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.551655 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:49.551710 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.551711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.553106 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.553107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.554162 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.554165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.555433 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.555434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.555499 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.555503 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.555562 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.555605 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260926 04:03:49.555728 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.551182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.0.200.130:39947)
0926 04:03:49.551411 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.551416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.551427 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:03:49.551472 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.551474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.551490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.551572 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:49.551789 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.552984 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.554280 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.555050 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.555308 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.555367 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.555596 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.555651 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260926 04:03:49.555948 11461 tablet_service.cc:1467] Processing CreateTablet for tablet 847b8e10703241a29b341ed7b135fb69 (DEFAULT_TABLE table=test-table [id=8caf325ed5a048baa81a5051a4ef3875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:49.556368 11461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847b8e10703241a29b341ed7b135fb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:49.556627 11327 tablet_service.cc:1467] Processing CreateTablet for tablet 847b8e10703241a29b341ed7b135fb69 (DEFAULT_TABLE table=test-table [id=8caf325ed5a048baa81a5051a4ef3875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:49.557019 11327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847b8e10703241a29b341ed7b135fb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:49.557449 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.550877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37435 (local address 127.0.0.1:40450)
0926 04:03:49.551453 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.551463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.551505 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.551788 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.551789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.551791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:49.551853 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.551855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.554271 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.554272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.555957 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.555960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.557172 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.557173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.557253 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.557259 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.557328 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.557373 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":84,"threads_started":1}
I20260926 04:03:49.557651 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.551390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.0.200.129:37435)
0926 04:03:49.551578 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.551583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.551593 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:03:49.551629 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.551631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.551646 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.551717 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:49.551929 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.552997 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.556053 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.556984 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.557070 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.557125 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.557361 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.557420 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260926 04:03:49.559316 11192 tablet_service.cc:1467] Processing CreateTablet for tablet 847b8e10703241a29b341ed7b135fb69 (DEFAULT_TABLE table=test-table [id=8caf325ed5a048baa81a5051a4ef3875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:03:49.559649 11192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847b8e10703241a29b341ed7b135fb69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:03:49.560488 11544 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Bootstrap starting.
I20260926 04:03:49.560551 11545 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Bootstrap starting.
I20260926 04:03:49.561261 11545 tablet_bootstrap.cc:654] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:49.561288 11544 tablet_bootstrap.cc:654] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:49.562081 11544 log.cc:824] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:49.562348 11545 log.cc:824] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:49.563221 11545 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: No bootstrap required, opened a new log
I20260926 04:03:49.563287 11544 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: No bootstrap required, opened a new log
I20260926 04:03:49.563297 11545 ts_tablet_manager.cc:1397] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 04:03:49.563341 11544 ts_tablet_manager.cc:1397] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:03:49.563397 11548 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Bootstrap starting.
I20260926 04:03:49.563972 11548 tablet_bootstrap.cc:654] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Neither blocks nor log segments found. Creating new log.
I20260926 04:03:49.564373 11548 log.cc:824] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Log is configured to *not* fsync() on all Append() calls
I20260926 04:03:49.565018 11548 tablet_bootstrap.cc:492] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: No bootstrap required, opened a new log
I20260926 04:03:49.565073 11548 ts_tablet_manager.cc:1397] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:03:49.565344 11544 raft_consensus.cc:348] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.565344 11545 raft_consensus.cc:348] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.565542 11545 raft_consensus.cc:374] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:49.565542 11544 raft_consensus.cc:374] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:49.565582 11545 raft_consensus.cc:729] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac2e9d22f33424182530ec239024816, State: Initialized, Role: FOLLOWER
I20260926 04:03:49.565582 11544 raft_consensus.cc:729] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db53b0ef92cb46fda557280c8f6b5982, State: Initialized, Role: FOLLOWER
I20260926 04:03:49.565704 11545 consensus_queue.cc:260] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.565704 11544 consensus_queue.cc:260] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.566045 11527 heartbeater.cc:503] Master 127.0.200.190:36881 was elected leader, sending a full tablet report...
I20260926 04:03:49.566109 11544 ts_tablet_manager.cc:1428] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:03:49.566133 11545 ts_tablet_manager.cc:1428] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 04:03:49.566239 11393 heartbeater.cc:503] Master 127.0.200.190:36881 was elected leader, sending a full tablet report...
I20260926 04:03:49.568282 11548 raft_consensus.cc:348] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.568531 11548 raft_consensus.cc:374] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:03:49.568562 11548 raft_consensus.cc:729] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6126d691769b42a48700791e6b110e91, State: Initialized, Role: FOLLOWER
I20260926 04:03:49.568737 11548 consensus_queue.cc:260] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.569271 11548 ts_tablet_manager.cc:1428] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 04:03:49.569274 11259 heartbeater.cc:503] Master 127.0.200.190:36881 was elected leader, sending a full tablet report...
W20260926 04:03:49.616444 11394 tablet.cc:2369] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:03:49.700445 11260 tablet.cc:2369] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:03:49.773034 11528 tablet.cc:2369] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:03:49.948166 11552 raft_consensus.cc:482] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:03:49.948348 11552 raft_consensus.cc:504] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.948724 11552 leader_election.cc:288] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ac2e9d22f33424182530ec239024816 (127.0.200.130:39947), db53b0ef92cb46fda557280c8f6b5982 (127.0.200.131:36641)
I20260926 04:03:49.952956 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.949052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:49689 (local address 127.0.200.130:39947)
0926 04:03:49.949127 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.949129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.949207 (+ 78us) server_negotiation.cc:407] Connection header received
0926 04:03:49.949248 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.949249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.949259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.949336 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:49.949620 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.950368 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.951986 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.952545 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.952627 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.952636 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.952843 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.952881 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:03:49.952956 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.949088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:54303 (local address 127.0.200.131:36641)
0926 04:03:49.949129 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:49.949131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:49.949208 (+ 77us) server_negotiation.cc:407] Connection header received
0926 04:03:49.949248 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:49.949249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:49.949259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:49.949336 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:03:49.949628 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.950368 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.951955 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.952546 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.952628 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.952636 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:49.952844 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 04:03:49.952881 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:03:49.953007 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.948968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:39947 (local address 127.0.200.129:49689)
0926 04:03:49.949109 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.949123 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.949196 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.949357 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.949358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.949360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:49.949535 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.949537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.950487 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.950489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.951852 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.951855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.952654 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.952655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.952748 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.952755 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.952829 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.952880 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":128,"spinlock_wait_cycles":15616,"thread_start_us":97,"threads_started":1}
I20260926 04:03:49.953015 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:49.948970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:36641 (local address 127.0.200.129:54303)
0926 04:03:49.949139 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:49.949148 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:03:49.949196 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:03:49.949357 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:03:49.949358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:03:49.949361 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:03:49.949533 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.949536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.950489 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.950489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.951852 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:03:49.951855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:03:49.952656 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:03:49.952656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:03:49.952748 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:49.952757 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 04:03:49.952829 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 04:03:49.952880 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":74,"thread_start_us":68,"threads_started":1}
I20260926 04:03:49.953423 11481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847b8e10703241a29b341ed7b135fb69" candidate_uuid: "6126d691769b42a48700791e6b110e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db53b0ef92cb46fda557280c8f6b5982" is_pre_election: true
I20260926 04:03:49.953359 11347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847b8e10703241a29b341ed7b135fb69" candidate_uuid: "6126d691769b42a48700791e6b110e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac2e9d22f33424182530ec239024816" is_pre_election: true
I20260926 04:03:49.953584 11481 raft_consensus.cc:2454] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6126d691769b42a48700791e6b110e91 in term 0.
I20260926 04:03:49.953606 11347 raft_consensus.cc:2454] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6126d691769b42a48700791e6b110e91 in term 0.
I20260926 04:03:49.953857 11148 leader_election.cc:302] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6126d691769b42a48700791e6b110e91, db53b0ef92cb46fda557280c8f6b5982; no voters:
I20260926 04:03:49.954048 11552 raft_consensus.cc:2781] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:03:49.954133 11552 raft_consensus.cc:482] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:03:49.954169 11552 raft_consensus.cc:3037] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:49.954911 11552 raft_consensus.cc:504] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.955061 11552 leader_election.cc:288] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [CANDIDATE]: Term 1 election: Requested vote from peers 0ac2e9d22f33424182530ec239024816 (127.0.200.130:39947), db53b0ef92cb46fda557280c8f6b5982 (127.0.200.131:36641)
I20260926 04:03:49.955233 11481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847b8e10703241a29b341ed7b135fb69" candidate_uuid: "6126d691769b42a48700791e6b110e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db53b0ef92cb46fda557280c8f6b5982"
I20260926 04:03:49.955260 11347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847b8e10703241a29b341ed7b135fb69" candidate_uuid: "6126d691769b42a48700791e6b110e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac2e9d22f33424182530ec239024816"
I20260926 04:03:49.955312 11481 raft_consensus.cc:3037] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:49.955343 11347 raft_consensus.cc:3037] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:03:49.956074 11481 raft_consensus.cc:2454] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6126d691769b42a48700791e6b110e91 in term 1.
I20260926 04:03:49.956156 11347 raft_consensus.cc:2454] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6126d691769b42a48700791e6b110e91 in term 1.
I20260926 04:03:49.956286 11148 leader_election.cc:302] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6126d691769b42a48700791e6b110e91, db53b0ef92cb46fda557280c8f6b5982; no voters:
I20260926 04:03:49.956470 11552 raft_consensus.cc:2781] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:03:49.956662 11552 raft_consensus.cc:686] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6126d691769b42a48700791e6b110e91, State: Running, Role: LEADER
I20260926 04:03:49.956784 11552 consensus_queue.cc:237] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } }
I20260926 04:03:49.957650 11067 catalog_manager.cc:5495] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6126d691769b42a48700791e6b110e91 (127.0.200.129). New cstate: current_term: 1 leader_uuid: "6126d691769b42a48700791e6b110e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6126d691769b42a48700791e6b110e91" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 37435 } health_report { overall_health: HEALTHY } } }
I20260926 04:03:50.005134 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:50.002540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.0.200.129:37435)
0926 04:03:50.002611 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:03:50.002615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:03:50.002664 (+ 49us) server_negotiation.cc:407] Connection header received
0926 04:03:50.002732 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:03:50.002733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:03:50.002750 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:03:50.002816 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:03:50.002918 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:50.003445 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:50.004228 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:03:50.004516 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:03:50.004588 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:03:50.004779 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:03:50.004876 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:03:50.004954 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 04:03:50.004989 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 04:03:50.005008 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:03:50.007026 11481 raft_consensus.cc:1261] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 1 FOLLOWER]: Refusing update from remote peer 6126d691769b42a48700791e6b110e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:50.007026 11347 raft_consensus.cc:1261] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 1 FOLLOWER]: Refusing update from remote peer 6126d691769b42a48700791e6b110e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:03:50.007402 11555 consensus_queue.cc:1035] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db53b0ef92cb46fda557280c8f6b5982" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:50.007709 11552 consensus_queue.cc:1035] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac2e9d22f33424182530ec239024816" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:03:50.010342 11559 mvcc.cc:205] Tried to move back new op lower bound from 7333459681305632768 to 7333459681103495168. Current Snapshot: MvccSnapshot[applied={T|T < 7333459681305632768 or (T in {7333459681305632768})}]
I20260926 04:03:50.010496 11560 mvcc.cc:205] Tried to move back new op lower bound from 7333459681305632768 to 7333459681103495168. Current Snapshot: MvccSnapshot[applied={T|T < 7333459681305632768}]
I20260926 04:03:50.012301 11068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:50.012421 11068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:03:50.013164 11068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 703b54a6784b491c9d4a96deacb46f77: Sending DeleteTablet for 3 replicas of tablet 847b8e10703241a29b341ed7b135fb69
I20260926 04:03:50.013505 11192 tablet_service.cc:1514] Processing DeleteTablet for tablet 847b8e10703241a29b341ed7b135fb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:50 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:40450
I20260926 04:03:50.013504 11327 tablet_service.cc:1514] Processing DeleteTablet for tablet 847b8e10703241a29b341ed7b135fb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:50 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:37842
I20260926 04:03:50.013744 11567 tablet_replica.cc:330] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: stopping tablet replica
I20260926 04:03:50.013744 11461 tablet_service.cc:1514] Processing DeleteTablet for tablet 847b8e10703241a29b341ed7b135fb69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:03:50 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:52866
I20260926 04:03:50.013844 11567 raft_consensus.cc:2229] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:03:50.013970 11567 raft_consensus.cc:2258] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:50.013979 11568 tablet_replica.cc:330] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: stopping tablet replica
I20260926 04:03:50.014063 11568 raft_consensus.cc:2229] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:50.014189 11568 raft_consensus.cc:2258] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:50.014282 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 11130
I20260926 04:03:50.014400 11567 ts_tablet_manager.cc:1905] T 847b8e10703241a29b341ed7b135fb69 P 6126d691769b42a48700791e6b110e91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:50.014623 11568 ts_tablet_manager.cc:1905] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:50.016222 11568 ts_tablet_manager.cc:1918] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:50.016292 11568 log.cc:1196] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/847b8e10703241a29b341ed7b135fb69
I20260926 04:03:50.016407 11569 tablet_replica.cc:330] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: stopping tablet replica
I20260926 04:03:50.016530 11569 raft_consensus.cc:2229] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:03:50.016598 11568 ts_tablet_manager.cc:1939] T 847b8e10703241a29b341ed7b135fb69 P 0ac2e9d22f33424182530ec239024816: Deleting consensus metadata
I20260926 04:03:50.016640 11569 raft_consensus.cc:2258] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:03:50.017000 11055 catalog_manager.cc:4841] TS 0ac2e9d22f33424182530ec239024816 (127.0.200.130:39947): tablet 847b8e10703241a29b341ed7b135fb69 (table test-table [id=8caf325ed5a048baa81a5051a4ef3875]) successfully deleted
I20260926 04:03:50.017269 11569 ts_tablet_manager.cc:1905] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:03:50.018759 11569 ts_tablet_manager.cc:1918] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:03:50.018821 11569 log.cc:1196] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/847b8e10703241a29b341ed7b135fb69
I20260926 04:03:50.019111 11569 ts_tablet_manager.cc:1939] T 847b8e10703241a29b341ed7b135fb69 P db53b0ef92cb46fda557280c8f6b5982: Deleting consensus metadata
I20260926 04:03:50.019563 11054 catalog_manager.cc:4841] TS db53b0ef92cb46fda557280c8f6b5982 (127.0.200.131:36641): tablet 847b8e10703241a29b341ed7b135fb69 (table test-table [id=8caf325ed5a048baa81a5051a4ef3875]) successfully deleted
W20260926 04:03:50.020380 11053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:37435 (error 108)
I20260926 04:03:50.020936 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:03:50.020817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:37435 (local address 127.0.0.1:40468)
0926 04:03:50.020870 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 04:03:50.020907 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:37435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:03:50.020961 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 11264
W20260926 04:03:50.021085 11053 catalog_manager.cc:4584] TS 6126d691769b42a48700791e6b110e91 (127.0.200.129:37435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 847b8e10703241a29b341ed7b135fb69: Network error: Client connection negotiation failed: client connection to 127.0.200.129:37435: connect: Connection refused (error 111)
I20260926 04:03:50.026223 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 11398
I20260926 04:03:50.030856 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 11036
2026-09-26T04:03:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2325 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:03:50 dist-test-slave-j8c1 krb5kdc[11573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:03:50 dist-test-slave-j8c1 krb5kdc[11573](info): set up 2 sockets
Sep 26 04:03:50 dist-test-slave-j8c1 krb5kdc[11573](info): commencing operation
krb5kdc: starting...
W20260926 04:03:51.079715 802 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:03:51 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:03:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:03:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangeradmin_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangeradmin_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangerlookup_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangerlookup_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
I20260926 04:03:51.254256 802 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIcgGpu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/postgres -l logfile start
2026-09-26 04:03:54.035 UTC [11603] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 04:03:54.035 UTC [11603] LOG: listening on IPv4 address "127.0.200.148", port 59905
2026-09-26 04:03:54.040 UTC [11603] LOG: listening on Unix socket "/tmp/.s.PGSQL.59905"
2026-09-26 04:03:54.064 UTC [11606] LOG: database system was shut down at 2026-09-26 04:03:52 UTC
2026-09-26 04:03:54.070 UTC [11603] LOG: database system is ready to accept connections
I20260926 04:03:55.031607 802 mini_postgres.cc:96] Postgres bound to 59905
2026-09-26 04:03:55.037 UTC [11614] FATAL: database "slave" does not exist
127.0.200.148:59905 - accepting connections
I20260926 04:03:55.038393 802 mini_ranger.cc:161] Starting Ranger...
I20260926 04:03:55.054395 802 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 04:03:55.662308 802 mini_ranger.cc:87] Created ranger Postgres database
I20260926 04:03:55.662400 802 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin
2026-09-26 04:03:55,944 [I] DB FLAVOR :POSTGRES
2026-09-26 04:03:55,944 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 04:03:55,944 [I] Checking connection..
2026-09-26 04:03:55,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 04:03:56,215 [I] Checking connection passed.
2026-09-26 04:03:56,216 [I] --------- Verifying version history table ---------
2026-09-26 04:03:56,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:03:56,462 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 04:03:56,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:03:56,738 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 04:03:56,738 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 04:03:56,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 04:03:56.988 UTC [11678] WARNING: there is no transaction in progress
2026-09-26 04:03:56,995 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 04:03:56,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:03:57,249 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 04:03:57,249 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 04:03:57,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 04:03:57,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 04:03:57,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j8c1.c.gcp-upstream.internal','N') ;"
2026-09-26 04:03:57,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 04:03:58,265 [I] Table x_portal_user does not exist in database ranger
2026-09-26 04:03:58,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 04:03:58,513 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 04:03:58,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 04:03:58,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 04:03:59,011 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 04:03:59,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 04:04:00.071 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.090 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.109 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.127 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.146 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.167 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.232 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.240 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.257 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.268 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.280 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.295 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.306 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.314 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.323 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:00.331 UTC [11786] WARNING: there is no transaction in progress
2026-09-26 04:04:01,145 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 04:04:01,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 04:04:01,406 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 04:04:01,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 04:04:01,659 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 04:04:01,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 04:04:01,907 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 04:04:01,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 04:04:02,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 04:04:02,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j8c1.c.gcp-upstream.internal';"
2026-09-26 04:04:02,637 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 04:04:02,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 04:04:02,876 [I] DB_PATCHES have already been applied
I20260926 04:04:02.881719 802 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIcgGpu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 04:04:02.881795 802 mini_ranger.cc:193] Using host: 127.0.200.148
I20260926 04:04:02.885030 802 mini_ranger.cc:227] Ranger admin URL: http://127.0.200.148:33427
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:04:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 04:04:04.058570 802 mini_ranger.cc:160] Time spent starting Ranger: real 9.020s user 0.000s sys 0.010s
Sep 26, 2026 4:04:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:04:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 4:04:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:04:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:04:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 04:04:27.026404 802 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangerkms_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/rangerkms_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.200.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.200.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 04:04:27 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395467, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 04:04:27.111492 802 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 04:04:27.121408 802 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 04:04:28.149390 802 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 04:04:28.149492 802 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms
I20260926 04:04:28.149514 802 mini_ranger_kms.cc:227] Using postgres at 127.0.200.148:59905
2026-09-26 04:04:28,335 [I] DB FLAVOR :POSTGRES
2026-09-26 04:04:28,335 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 04:04:28,336 [I] Checking connection
2026-09-26 04:04:28,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 04:04:28,608 [I] connection success
2026-09-26 04:04:28,608 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 04:04:28,609 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 04:04:28,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 04:04:28,881 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 04:04:28,881 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 04:04:28,881 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 04:04:28,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:59905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 04:04:29,175 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 04:04:29.723627 802 mini_ranger_kms.cc:310] Created kms service
I20260926 04:04:29.987927 802 mini_ranger_kms.cc:326] Created kudu user
I20260926 04:04:30.038309 802 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 04:04:30 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395467, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:04:30.454965 802 mini_ranger_kms.cc:384] Added ranger policy
I20260926 04:04:30.455098 802 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 04:04:30.455137 802 mini_ranger_kms.cc:240] Using host: 127.0.200.148
I20260926 04:04:30.458602 802 mini_ranger_kms.cc:276] Ranger KMS PID: 11967
I20260926 04:04:30.458737 802 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.200.148:59097
Sep 26, 2026 4:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:04:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:04:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:04:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 04:04:32.764667 802 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.653s user 0.010s sys 0.157s
I20260926 04:04:32.764784 802 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 04:04:32.764868 802 mini_ranger_kms.cc:401] 127.0.200.148:59097/kms/v1/keys
Sep 26 04:04:32 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790395472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 04:04:33 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
Sep 26 04:04:33 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
Sep 26 04:04:36 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:04:36.090942 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44443
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:44443
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:59097/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:04:36.199163 12017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:04:36.199384 12017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:04:36.199412 12017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:04:36.200647 12017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:04:36.200699 12017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:04:36.200713 12017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:04:36.200727 12017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:04:36.200739 12017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:04:36.202347 12017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.200.148:59097/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:44443
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:04:36.202615 12017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:04:36.202884 12017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:04:36.209200 12023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:36.209232 12022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:04:36.209283 12017 server_base.cc:1034] running on GCE node
W20260926 04:04:36.209478 12025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:04:36.209753 12017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:04:36.210139 12017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:04:36.211331 12017 hybrid_clock.cc:648] HybridClock initialized: now 1790395476211304 us; error 43 us; skew 500 ppm
Sep 26 04:04:36 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:04:36.214351 12017 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:04:36.214941 12017 webserver.cc:460] Webserver started at http://127.0.200.190:38095/ using document root <none> and password file <none>
I20260926 04:04:36.215201 12017 fs_manager.cc:359] Metadata directory not provided
I20260926 04:04:36.215268 12017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:04:36.215442 12017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:04:36 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:04:36.265959 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "b552299aaf4b4e4ebd787d1b06c5057e"
format_stamp: "Formatted at 2026-09-26 04:04:36 on dist-test-slave-j8c1"
server_key: "acaa17511a58392c362e21d4233869b2"
server_key_iv: "c66d0f2a53e0475543d92750d64fa9c2"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:36.266423 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "b552299aaf4b4e4ebd787d1b06c5057e"
format_stamp: "Formatted at 2026-09-26 04:04:36 on dist-test-slave-j8c1"
server_key: "acaa17511a58392c362e21d4233869b2"
server_key_iv: "c66d0f2a53e0475543d92750d64fa9c2"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:36.268164 12017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:04:36.281502 12032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:36.281733 12017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:04:36.281826 12017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "b552299aaf4b4e4ebd787d1b06c5057e"
format_stamp: "Formatted at 2026-09-26 04:04:36 on dist-test-slave-j8c1"
server_key: "acaa17511a58392c362e21d4233869b2"
server_key_iv: "c66d0f2a53e0475543d92750d64fa9c2"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:36.281965 12017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:04:36.298995 12017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:04:36.301225 12017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:04:36.301414 12017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:04:36.306278 12017 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:44443
I20260926 04:04:36.306301 12084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:44443 every 8 connection(s)
I20260926 04:04:36.306854 12017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:04:36.307549 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:04:36.308287 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 12017
I20260926 04:04:36.308405 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:04:36.308595 802 external_mini_cluster.cc:1428] Setting key 86803d7b307213061c040bfe09124398
I20260926 04:04:36.311182 12085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: Bootstrap starting.
I20260926 04:04:36.311920 12085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: Neither blocks nor log segments found. Creating new log.
I20260926 04:04:36.312201 12085 log.cc:824] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: Log is configured to *not* fsync() on all Append() calls
I20260926 04:04:36.312987 12085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: No bootstrap required, opened a new log
I20260926 04:04:36.315404 12085 raft_consensus.cc:348] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } }
I20260926 04:04:36.315573 12085 raft_consensus.cc:374] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:04:36.315637 12085 raft_consensus.cc:729] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b552299aaf4b4e4ebd787d1b06c5057e, State: Initialized, Role: FOLLOWER
I20260926 04:04:36.315824 12085 consensus_queue.cc:260] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } }
I20260926 04:04:36.315922 12085 raft_consensus.cc:388] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:04:36.315976 12085 raft_consensus.cc:482] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:04:36.316033 12085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:04:36.316711 12085 raft_consensus.cc:504] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } }
I20260926 04:04:36.316830 12085 leader_election.cc:302] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b552299aaf4b4e4ebd787d1b06c5057e; no voters:
Sep 26 04:04:36 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:04:36.317054 12085 leader_election.cc:288] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:04:36.317108 12090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:04:36.317317 12090 raft_consensus.cc:686] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [term 1 LEADER]: Becoming Leader. State: Replica: b552299aaf4b4e4ebd787d1b06c5057e, State: Running, Role: LEADER
I20260926 04:04:36.317346 12085 sys_catalog.cc:564] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:04:36.317498 12090 consensus_queue.cc:237] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } }
I20260926 04:04:36.318262 12090 sys_catalog.cc:455] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b552299aaf4b4e4ebd787d1b06c5057e. Latest consensus state: current_term: 1 leader_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } } }
I20260926 04:04:36.318475 12090 sys_catalog.cc:458] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [sys.catalog]: This master's current role is: LEADER
I20260926 04:04:36.318737 12092 sys_catalog.cc:455] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b552299aaf4b4e4ebd787d1b06c5057e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44443 } } }
I20260926 04:04:36.318868 12092 sys_catalog.cc:458] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e [sys.catalog]: This master's current role is: LEADER
I20260926 04:04:36.319032 12099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:04:36.319244 12085 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 04:04:36.319595 12099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:04:36.325201 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:36.309571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.0.200.190:44443)
0926 04:04:36.309754 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:36.309764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:36.309801 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:04:36.310412 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:36.310421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:36.310488 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:36.310660 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:04:36.311387 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:36.312730 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:36.313794 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:36.314522 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:36.314666 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:36.317672 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:04:36.317677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:04:36.317677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:04:36.317690 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:04:36.319936 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:36.320381 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:36.320384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:36.320385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:36.320479 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:36.320719 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:36.320720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:36.320720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:36.322902 (+ 2182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:04:36.323824 (+ 922us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:36.324300 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 04:04:36.324841 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260926 04:04:36.332188 12099 catalog_manager.cc:1347] Generated new cluster ID: 782a7473b33c4126b1f9cab14e716a06
I20260926 04:04:36.332252 12099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:04:36.341334 12099 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:04:36.342118 12099 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:04:36.349882 12099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: Generated new TSK 0
I20260926 04:04:36.350185 12099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:04:37.179111 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:59097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:04:37.291258 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:04:37.291514 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:04:37.291554 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:04:37.292889 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:04:37.292937 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:04:37.292975 12130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:04:37.294499 12130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.200.148:59097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:04:37.294787 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:04:37.295022 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:04:37.300318 12138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.300318 12136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.300643 12135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:04:37.301143 12130 server_base.cc:1034] running on GCE node
I20260926 04:04:37.301395 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:04:37.301693 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:04:37.302850 12130 hybrid_clock.cc:648] HybridClock initialized: now 1790395477302834 us; error 32 us; skew 500 ppm
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:04:37.305768 12130 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:04:37.306347 12130 webserver.cc:460] Webserver started at http://127.0.200.129:34457/ using document root <none> and password file <none>
I20260926 04:04:37.306633 12130 fs_manager.cc:359] Metadata directory not provided
I20260926 04:04:37.306697 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:04:37.306865 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:04:37.320992 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "afba6b54a97f46d195fc37a23cc1c43c"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "1444d5a4a89727180b8cb90aa3ca3fee"
server_key_iv: "9897b2832f2fae96ddbf97e75f43df36"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.321557 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "afba6b54a97f46d195fc37a23cc1c43c"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "1444d5a4a89727180b8cb90aa3ca3fee"
server_key_iv: "9897b2832f2fae96ddbf97e75f43df36"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.323804 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 04:04:37.337047 12145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.337260 12130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:04:37.337363 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "afba6b54a97f46d195fc37a23cc1c43c"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "1444d5a4a89727180b8cb90aa3ca3fee"
server_key_iv: "9897b2832f2fae96ddbf97e75f43df36"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.337491 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:04:37.365852 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:04:37.368433 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:04:37.368650 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:04:37.368952 12130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:04:37.369343 12130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:04:37.369426 12130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.369540 12130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:04:37.369614 12130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.377404 12130 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:41007
I20260926 04:04:37.377781 12260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:41007 every 8 connection(s)
I20260926 04:04:37.378049 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:04:37.379716 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 12130
I20260926 04:04:37.379817 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:04:37.380020 802 external_mini_cluster.cc:1428] Setting key 3e6eff8e82bd0d3221a6932089e015c4
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:04:37.392261 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.378919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44443 (local address 127.0.200.129:42497)
0926 04:04:37.379108 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.379160 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.379845 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.381511 (+ 1666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.381516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.381779 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:04:37.382087 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.382095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.383214 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.383215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.384582 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.384585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.385209 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.385210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.385361 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.386086 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:04:37.386098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:04:37.388322 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:04:37.390483 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.390491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.390494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.390819 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.391046 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.391046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.391047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.391191 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.391525 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:04:37.391532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:04:37.391708 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 04:04:37.391824 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 04:04:37.392001 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260926 04:04:37.392866 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.378996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:42497 (local address 127.0.200.190:44443)
0926 04:04:37.381263 (+ 2267us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.381265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.381275 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:04:37.381300 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.381301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.381315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.381397 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:04:37.382212 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.383096 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.384685 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.385069 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.385136 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.388440 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:04:37.388442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:04:37.388442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:04:37.388463 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:04:37.390373 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.390906 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.390907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.390908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.390973 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.391265 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.391266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.391266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.391416 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:04:37.391497 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.392669 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.392766 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2243,"thread_start_us":74,"threads_started":1}
I20260926 04:04:37.393741 12261 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44443
I20260926 04:04:37.393848 12261 heartbeater.cc:464] Registering TS with master...
I20260926 04:04:37.394038 12261 heartbeater.cc:511] Master 127.0.200.190:44443 requested a full tablet report, sending...
I20260926 04:04:37.394678 12049 ts_manager.cc:194] Registered new tserver with Master: afba6b54a97f46d195fc37a23cc1c43c (127.0.200.129:41007)
I20260926 04:04:37.395612 12049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:42497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:04:37.418449 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:59097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:04:37.560717 12267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:04:37.561074 12267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:04:37.561165 12267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:04:37.563199 12267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:04:37.563316 12267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:04:37.563393 12267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:04:37.565762 12267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.200.148:59097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:04:37.566241 12267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:04:37.566572 12267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:04:37.573607 12275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.573693 12274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.573746 12277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:04:37.574698 12267 server_base.cc:1034] running on GCE node
I20260926 04:04:37.574913 12267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:04:37.575232 12267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:04:37.576421 12267 hybrid_clock.cc:648] HybridClock initialized: now 1790395477576397 us; error 34 us; skew 500 ppm
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:04:37.579692 12267 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:04:37.580222 12267 webserver.cc:460] Webserver started at http://127.0.200.130:36823/ using document root <none> and password file <none>
I20260926 04:04:37.580463 12267 fs_manager.cc:359] Metadata directory not provided
I20260926 04:04:37.580510 12267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:04:37.580646 12267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:04:37.598875 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "2635a0209a864ed7b981887e275e000b"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "aba201174b7c192d0a8e78b68fdbd495"
server_key_iv: "ade1b97af6023a7ef9240c4199e48603"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.599486 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "2635a0209a864ed7b981887e275e000b"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "aba201174b7c192d0a8e78b68fdbd495"
server_key_iv: "ade1b97af6023a7ef9240c4199e48603"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.603834 12267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260926 04:04:37.626721 12284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.627005 12267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 04:04:37.627095 12267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "2635a0209a864ed7b981887e275e000b"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "aba201174b7c192d0a8e78b68fdbd495"
server_key_iv: "ade1b97af6023a7ef9240c4199e48603"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.627210 12267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:04:37.656857 12267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:04:37.659374 12267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:04:37.659585 12267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:04:37.659857 12267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:04:37.660282 12267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:04:37.660369 12267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.660436 12267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:04:37.660490 12267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.669392 12267 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:43115
I20260926 04:04:37.669884 12267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:04:37.670574 12398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:43115 every 8 connection(s)
I20260926 04:04:37.676576 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 12267
I20260926 04:04:37.676673 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:04:37.676856 802 external_mini_cluster.cc:1428] Setting key 81882b3d6156330720a4529ca5f1febf
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:04:37.685094 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.671291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:35645 (local address 127.0.200.190:44443)
0926 04:04:37.671357 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.671360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.672160 (+ 800us) server_negotiation.cc:407] Connection header received
0926 04:04:37.672823 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.672825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.672842 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.672872 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:04:37.673679 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.674378 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.675984 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.676297 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.676343 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.681794 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:04:37.681796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:04:37.681796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:04:37.681804 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:04:37.683464 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.683952 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.683953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.683954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.684016 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.684305 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.684305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.684305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.684439 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:04:37.684478 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.684888 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.684982 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:04:37.685900 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.671294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44443 (local address 127.0.200.130:35645)
0926 04:04:37.672008 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.672069 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.672702 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.673062 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.673067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.673303 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:04:37.673569 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.673576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.674501 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.674501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.675858 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.675861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.679203 (+ 3342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.679203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.679369 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.680019 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:04:37.680047 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:04:37.681675 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:04:37.683567 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.683574 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.683577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.683866 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.684105 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.684106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.684106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.684237 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.684520 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:04:37.684526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:04:37.684739 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 04:04:37.685487 (+ 748us) client_negotiation.cc:240] Negotiation successful
0926 04:04:37.685663 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":86,"threads_started":1}
I20260926 04:04:37.686615 12399 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44443
I20260926 04:04:37.686726 12399 heartbeater.cc:464] Registering TS with master...
I20260926 04:04:37.686933 12399 heartbeater.cc:511] Master 127.0.200.190:44443 requested a full tablet report, sending...
I20260926 04:04:37.687295 12049 ts_manager.cc:194] Registered new tserver with Master: 2635a0209a864ed7b981887e275e000b (127.0.200.130:43115)
I20260926 04:04:37.687824 12049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:35645
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:04:37.714665 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:59097/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:04:37.836493 12404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:04:37.836865 12404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:04:37.836962 12404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:04:37.838804 12404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:04:37.839032 12404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:04:37.839144 12404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:04:37.841463 12404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.200.148:59097/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:04:37.841922 12404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:04:37.842314 12404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:04:37.849493 12410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.849912 12409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:04:37.850168 12412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:04:37.851329 12404 server_base.cc:1034] running on GCE node
I20260926 04:04:37.851539 12404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:04:37.851810 12404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:04:37.852988 12404 hybrid_clock.cc:648] HybridClock initialized: now 1790395477852966 us; error 36 us; skew 500 ppm
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:04:37.856067 12404 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:04:37.856582 12404 webserver.cc:460] Webserver started at http://127.0.200.131:43803/ using document root <none> and password file <none>
I20260926 04:04:37.856843 12404 fs_manager.cc:359] Metadata directory not provided
I20260926 04:04:37.856899 12404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:04:37.857029 12404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:04:37.877907 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "10c2d9bf5ae62b72a53628791e420280"
server_key_iv: "41f2b54033f3b6577bd44f4b4b9ae5d8"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.878394 12404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "10c2d9bf5ae62b72a53628791e420280"
server_key_iv: "41f2b54033f3b6577bd44f4b4b9ae5d8"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.880398 12404 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260926 04:04:37.896266 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.896488 12404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 04:04:37.896577 12404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed"
format_stamp: "Formatted at 2026-09-26 04:04:37 on dist-test-slave-j8c1"
server_key: "10c2d9bf5ae62b72a53628791e420280"
server_key_iv: "41f2b54033f3b6577bd44f4b4b9ae5d8"
server_key_version: "kuduclusterkey@0"
I20260926 04:04:37.896677 12404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:04:37.920739 12404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:04:37.927093 12404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:04:37.927291 12404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:04:37.927565 12404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:04:37.928010 12404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:04:37.928067 12404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.928138 12404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:04:37.928166 12404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:04:37.936105 12404 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:35689
I20260926 04:04:37.936422 12532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:35689 every 8 connection(s)
I20260926 04:04:37.936678 12404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:04:37.942793 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 12404
I20260926 04:04:37.942922 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:04:37.943106 802 external_mini_cluster.cc:1428] Setting key 3ae8f39570cc01588f1c0253346828aa
Sep 26 04:04:37 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:04:37.949803 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.937632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:34759 (local address 127.0.200.190:44443)
0926 04:04:37.937679 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.937682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.937874 (+ 192us) server_negotiation.cc:407] Connection header received
0926 04:04:37.938406 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.938407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.938420 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.938516 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:04:37.940798 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.941633 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.943272 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.943566 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.943620 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.946347 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:04:37.946349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:04:37.946349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:04:37.946358 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:04:37.948161 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.948679 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.948680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.948681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.948741 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.949035 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.949036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.949036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.949189 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:04:37.949233 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.949609 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.949698 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:04:37.950328 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.937460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44443 (local address 127.0.200.131:34759)
0926 04:04:37.937746 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.937780 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.938295 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.938674 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.938679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.938980 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:04:37.940653 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.940662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.941788 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.941789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.943156 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.943159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.943723 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.943724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.943874 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.944534 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:04:37.944547 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:04:37.946226 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:04:37.948262 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.948269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.948272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.948583 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.948815 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:04:37.948816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:04:37.948817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:04:37.948953 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:04:37.949280 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:04:37.949287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:04:37.949453 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 04:04:37.949910 (+ 457us) client_negotiation.cc:240] Negotiation successful
0926 04:04:37.950077 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":83,"threads_started":1}
I20260926 04:04:37.950886 12533 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44443
I20260926 04:04:37.950994 12533 heartbeater.cc:464] Registering TS with master...
I20260926 04:04:37.951211 12533 heartbeater.cc:511] Master 127.0.200.190:44443 requested a full tablet report, sending...
I20260926 04:04:37.951567 12049 ts_manager.cc:194] Registered new tserver with Master: d0c197df5dff4dcd9f49ecb62f4a4aed (127.0.200.131:35689)
I20260926 04:04:37.952059 12049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:34759
I20260926 04:04:37.957207 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:04:37.976351 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.959229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56716 (local address 127.0.200.190:44443)
0926 04:04:37.959292 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.959294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.960516 (+ 1222us) server_negotiation.cc:407] Connection header received
0926 04:04:37.966670 (+ 6154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.966672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.966686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.966718 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:04:37.967572 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.968347 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.971046 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.971357 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.971410 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.973142 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:04:37.973144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:04:37.973144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:04:37.973153 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:04:37.974957 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.975408 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.975409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.975410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.975469 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:04:37.975697 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:04:37.975698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:04:37.975698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:04:37.975836 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:04:37.975915 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.976163 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.976247 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:04:37.978377 12049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:04:37.980549 12049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:04:37.993711 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.984034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41007 (local address 127.0.0.1:33372)
0926 04:04:37.985810 (+ 1776us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.985819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.985861 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.986935 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.986939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.986942 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:04:37.987012 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.989559 (+ 2547us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.990846 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.990846 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.992416 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.992418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.993409 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.993410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.993486 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.993499 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 04:04:37.993566 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 04:04:37.993619 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1762,"thread_start_us":81,"threads_started":1}
I20260926 04:04:37.994180 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.984536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.0.200.129:41007)
0926 04:04:37.985952 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.985956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.985965 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:04:37.985999 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.986001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.986017 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.986084 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:04:37.989640 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.990751 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.992500 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.993239 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.993310 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.993374 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.994050 (+ 676us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.994096 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":70,"threads_started":1}
I20260926 04:04:37.995128 12192 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe6fc1b77854c688253c4d828566629 (DEFAULT_TABLE table=test-table [id=e11452365f4d4bb68dd5e12111f10ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:04:37.995535 12192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe6fc1b77854c688253c4d828566629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:04:37.998185 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.984845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43115 (local address 127.0.0.1:45200)
0926 04:04:37.988313 (+ 3468us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.988324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.988378 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.988638 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.988638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.988641 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:04:37.988689 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.988691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.995229 (+ 6538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.995230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.996341 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.996343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.997935 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.997936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:37.997998 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.998003 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:04:37.998055 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:04:37.998096 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3450,"thread_start_us":63,"threads_started":1}
I20260926 04:04:37.998567 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.985085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.0.200.130:43115)
0926 04:04:37.986223 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.986227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.988439 (+ 2212us) server_negotiation.cc:407] Connection header received
0926 04:04:37.988482 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.988484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.988498 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.988562 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:04:37.988762 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.995122 (+ 6360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.996949 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.997747 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:37.997823 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:37.997897 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:37.998402 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 04:04:37.998445 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":102,"threads_started":1}
I20260926 04:04:37.999382 12332 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe6fc1b77854c688253c4d828566629 (DEFAULT_TABLE table=test-table [id=e11452365f4d4bb68dd5e12111f10ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:04:37.999719 12332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe6fc1b77854c688253c4d828566629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:04:38.006033 12553 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Bootstrap starting.
I20260926 04:04:38.007102 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.983830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35689 (local address 127.0.0.1:50054)
0926 04:04:37.985485 (+ 1655us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:37.985528 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:37.985589 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:37.987298 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:37.987299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:37.987301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:04:37.987362 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:37.987364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.004588 (+ 17224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.004589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.005756 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:38.005760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.006827 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.006828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.006904 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.006909 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:04:38.006974 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 04:04:38.007021 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":63,"threads_started":1}
I20260926 04:04:38.007119 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:37.984340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.0.200.131:35689)
0926 04:04:37.985728 (+ 1388us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:37.985733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:37.986245 (+ 512us) server_negotiation.cc:407] Connection header received
0926 04:04:37.986289 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:37.986291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:37.986306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:37.986376 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:04:37.987438 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:37.996815 (+ 9377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.005897 (+ 9082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.006733 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.006825 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.006903 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:38.006987 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 04:04:38.007024 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":85,"threads_started":1}
I20260926 04:04:38.008436 12464 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe6fc1b77854c688253c4d828566629 (DEFAULT_TABLE table=test-table [id=e11452365f4d4bb68dd5e12111f10ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:04:38.008781 12464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe6fc1b77854c688253c4d828566629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:04:38.012501 12552 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Bootstrap starting.
I20260926 04:04:38.013147 12553 tablet_bootstrap.cc:654] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Neither blocks nor log segments found. Creating new log.
I20260926 04:04:38.013811 12553 log.cc:824] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:04:38.014734 12552 tablet_bootstrap.cc:654] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Neither blocks nor log segments found. Creating new log.
I20260926 04:04:38.015333 12552 log.cc:824] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:04:38.015748 12554 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Bootstrap starting.
I20260926 04:04:38.016829 12554 tablet_bootstrap.cc:654] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Neither blocks nor log segments found. Creating new log.
I20260926 04:04:38.017225 12554 log.cc:824] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Log is configured to *not* fsync() on all Append() calls
I20260926 04:04:38.018127 12553 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: No bootstrap required, opened a new log
I20260926 04:04:38.018270 12553 ts_tablet_manager.cc:1397] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260926 04:04:38.021720 12553 raft_consensus.cc:348] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.022034 12553 raft_consensus.cc:374] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:04:38.022121 12553 raft_consensus.cc:729] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afba6b54a97f46d195fc37a23cc1c43c, State: Initialized, Role: FOLLOWER
I20260926 04:04:38.022349 12553 consensus_queue.cc:260] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.022751 12553 ts_tablet_manager.cc:1428] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 04:04:38.022915 12261 heartbeater.cc:503] Master 127.0.200.190:44443 was elected leader, sending a full tablet report...
I20260926 04:04:38.024541 12552 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: No bootstrap required, opened a new log
I20260926 04:04:38.024667 12552 ts_tablet_manager.cc:1397] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260926 04:04:38.026624 12552 raft_consensus.cc:348] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.026859 12552 raft_consensus.cc:374] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:04:38.027341 12552 raft_consensus.cc:729] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2635a0209a864ed7b981887e275e000b, State: Initialized, Role: FOLLOWER
I20260926 04:04:38.027546 12552 consensus_queue.cc:260] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.027874 12552 ts_tablet_manager.cc:1428] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:04:38.028021 12399 heartbeater.cc:503] Master 127.0.200.190:44443 was elected leader, sending a full tablet report...
I20260926 04:04:38.029852 12554 tablet_bootstrap.cc:492] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: No bootstrap required, opened a new log
I20260926 04:04:38.029996 12554 ts_tablet_manager.cc:1397] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260926 04:04:38.032153 12554 raft_consensus.cc:348] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.032390 12554 raft_consensus.cc:374] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:04:38.032461 12554 raft_consensus.cc:729] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c197df5dff4dcd9f49ecb62f4a4aed, State: Initialized, Role: FOLLOWER
I20260926 04:04:38.032639 12554 consensus_queue.cc:260] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.033043 12554 ts_tablet_manager.cc:1428] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 04:04:38.033241 12533 heartbeater.cc:503] Master 127.0.200.190:44443 was elected leader, sending a full tablet report...
Sep 26 04:04:38 dist-test-slave-j8c1 krb5kdc[11573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
W20260926 04:04:38.129222 12262 tablet.cc:2369] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:04:38.134150 12559 raft_consensus.cc:482] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:04:38.134295 12559 raft_consensus.cc:504] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.134728 12559 leader_election.cc:288] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0c197df5dff4dcd9f49ecb62f4a4aed (127.0.200.131:35689), afba6b54a97f46d195fc37a23cc1c43c (127.0.200.129:41007)
I20260926 04:04:38.138854 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.134870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35689 (local address 127.0.200.130:33027)
0926 04:04:38.134995 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:38.135007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:38.135047 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:38.135333 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:38.135334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:38.135336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:04:38.135401 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:38.135403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.136507 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.136508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.137638 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:38.137642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.138533 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.138534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.138660 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.138665 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:04:38.138717 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:04:38.138766 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260926 04:04:38.139065 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.135153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:33027 (local address 127.0.200.131:35689)
0926 04:04:38.135208 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:38.135210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:38.135219 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:04:38.135250 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:38.135251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:38.135263 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:38.135309 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:04:38.135468 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.136392 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.137742 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.138253 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.138331 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.138341 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:38.138958 (+ 617us) server_negotiation.cc:299] Negotiation successful
0926 04:04:38.139005 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 04:04:38.139622 12478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe6fc1b77854c688253c4d828566629" candidate_uuid: "2635a0209a864ed7b981887e275e000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" is_pre_election: true
I20260926 04:04:38.139806 12478 raft_consensus.cc:2454] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2635a0209a864ed7b981887e275e000b in term 0.
I20260926 04:04:38.140030 12286 leader_election.cc:302] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2635a0209a864ed7b981887e275e000b, d0c197df5dff4dcd9f49ecb62f4a4aed; no voters:
I20260926 04:04:38.140347 12559 raft_consensus.cc:2781] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:04:38.140415 12559 raft_consensus.cc:482] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:04:38.140436 12559 raft_consensus.cc:3037] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:04:38.141806 12559 raft_consensus.cc:504] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.141945 12559 leader_election.cc:288] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [CANDIDATE]: Term 1 election: Requested vote from peers d0c197df5dff4dcd9f49ecb62f4a4aed (127.0.200.131:35689), afba6b54a97f46d195fc37a23cc1c43c (127.0.200.129:41007)
I20260926 04:04:38.142335 12478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe6fc1b77854c688253c4d828566629" candidate_uuid: "2635a0209a864ed7b981887e275e000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed"
I20260926 04:04:38.142437 12478 raft_consensus.cc:3037] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:04:38.143289 12478 raft_consensus.cc:2454] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2635a0209a864ed7b981887e275e000b in term 1.
I20260926 04:04:38.143467 12286 leader_election.cc:302] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2635a0209a864ed7b981887e275e000b, d0c197df5dff4dcd9f49ecb62f4a4aed; no voters:
I20260926 04:04:38.143606 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.135093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:52261 (local address 127.0.200.129:41007)
0926 04:04:38.135134 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:38.135137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:38.135289 (+ 152us) server_negotiation.cc:407] Connection header received
0926 04:04:38.138151 (+ 2862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:38.138153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:38.138162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:38.138202 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:04:38.138426 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.139135 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.141800 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.142344 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.142393 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.142401 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:38.143479 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0926 04:04:38.143534 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:04:38.143989 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.134933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41007 (local address 127.0.200.130:52261)
0926 04:04:38.135230 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:38.135239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:04:38.138063 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:04:38.138264 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:04:38.138265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:04:38.138267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:04:38.138357 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:38.138358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.140033 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.140034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.141704 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:04:38.141707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.143266 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.143267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:04:38.143392 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.143396 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:04:38.143861 (+ 465us) client_negotiation.cc:240] Negotiation successful
0926 04:04:38.143904 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":65,"threads_started":1}
I20260926 04:04:38.144726 12206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe6fc1b77854c688253c4d828566629" candidate_uuid: "2635a0209a864ed7b981887e275e000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afba6b54a97f46d195fc37a23cc1c43c" is_pre_election: true
I20260926 04:04:38.144893 12206 raft_consensus.cc:2454] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2635a0209a864ed7b981887e275e000b in term 0.
I20260926 04:04:38.145017 12211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe6fc1b77854c688253c4d828566629" candidate_uuid: "2635a0209a864ed7b981887e275e000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afba6b54a97f46d195fc37a23cc1c43c"
I20260926 04:04:38.145083 12211 raft_consensus.cc:3037] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:04:38.145938 12211 raft_consensus.cc:2454] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2635a0209a864ed7b981887e275e000b in term 1.
I20260926 04:04:38.146417 12559 raft_consensus.cc:2781] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:04:38.146620 12559 raft_consensus.cc:686] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 1 LEADER]: Becoming Leader. State: Replica: 2635a0209a864ed7b981887e275e000b, State: Running, Role: LEADER
I20260926 04:04:38.146733 12559 consensus_queue.cc:237] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } }
I20260926 04:04:38.147935 12049 catalog_manager.cc:5495] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2635a0209a864ed7b981887e275e000b (127.0.200.130). New cstate: current_term: 1 leader_uuid: "2635a0209a864ed7b981887e275e000b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2635a0209a864ed7b981887e275e000b" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 43115 } health_report { overall_health: HEALTHY } } }
W20260926 04:04:38.171509 12400 tablet.cc:2369] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:04:38.186847 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.183002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.0.200.130:43115)
0926 04:04:38.183055 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:04:38.183057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:04:38.183110 (+ 53us) server_negotiation.cc:407] Connection header received
0926 04:04:38.183246 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:04:38.183247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:04:38.183276 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:04:38.183370 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:04:38.183568 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.184307 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.185472 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:04:38.185788 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:04:38.185855 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:04:38.186070 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:04:38.186214 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:04:38.186308 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 04:04:38.186726 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 04:04:38.186748 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 04:04:38.187669 12534 tablet.cc:2369] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:04:38.203797 12211 raft_consensus.cc:1261] T 9fe6fc1b77854c688253c4d828566629 P afba6b54a97f46d195fc37a23cc1c43c [term 1 FOLLOWER]: Refusing update from remote peer 2635a0209a864ed7b981887e275e000b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:04:38.204216 12559 consensus_queue.cc:1035] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [LEADER]: Connected to new peer: Peer: permanent_uuid: "afba6b54a97f46d195fc37a23cc1c43c" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:04:38.206872 12478 raft_consensus.cc:1261] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 1 FOLLOWER]: Refusing update from remote peer 2635a0209a864ed7b981887e275e000b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:04:38.207188 12559 consensus_queue.cc:1035] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0c197df5dff4dcd9f49ecb62f4a4aed" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:04:38.217082 12566 mvcc.cc:205] Tried to move back new op lower bound from 7333459878719356928 to 7333459878489690112. Current Snapshot: MvccSnapshot[applied={T|T < 7333459878719356928}]
I20260926 04:04:38.220196 12568 mvcc.cc:205] Tried to move back new op lower bound from 7333459878719356928 to 7333459878489690112. Current Snapshot: MvccSnapshot[applied={T|T < 7333459878719356928}]
I20260926 04:04:38.227552 12048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:04:38.227691 12048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:04:38.228469 12048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b552299aaf4b4e4ebd787d1b06c5057e: Sending DeleteTablet for 3 replicas of tablet 9fe6fc1b77854c688253c4d828566629
I20260926 04:04:38.229017 12464 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe6fc1b77854c688253c4d828566629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:04:38 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:50054
I20260926 04:04:38.229274 12192 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe6fc1b77854c688253c4d828566629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:04:38 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:33372
I20260926 04:04:38.229647 12332 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe6fc1b77854c688253c4d828566629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:04:38 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45200
I20260926 04:04:38.230130 12576 tablet_replica.cc:330] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: stopping tablet replica
I20260926 04:04:38.230248 12576 raft_consensus.cc:2229] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:04:38.230258 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 12130
I20260926 04:04:38.231102 12576 raft_consensus.cc:2258] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:04:38.231312 12574 tablet_replica.cc:330] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: stopping tablet replica
I20260926 04:04:38.231422 12574 raft_consensus.cc:2229] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:04:38.231544 12574 raft_consensus.cc:2258] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:04:38.232156 12576 ts_tablet_manager.cc:1905] T 9fe6fc1b77854c688253c4d828566629 P 2635a0209a864ed7b981887e275e000b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:04:38.236847 12574 ts_tablet_manager.cc:1905] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 04:04:38.238492 12035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:41007 (error 108)
I20260926 04:04:38.238922 12574 ts_tablet_manager.cc:1918] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:04:38.239015 12574 log.cc:1196] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/9fe6fc1b77854c688253c4d828566629
I20260926 04:04:38.239315 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 12267
I20260926 04:04:38.240940 12574 ts_tablet_manager.cc:1939] T 9fe6fc1b77854c688253c4d828566629 P d0c197df5dff4dcd9f49ecb62f4a4aed: Deleting consensus metadata
I20260926 04:04:38.242139 12034 catalog_manager.cc:4841] TS d0c197df5dff4dcd9f49ecb62f4a4aed (127.0.200.131:35689): tablet 9fe6fc1b77854c688253c4d828566629 (table test-table [id=e11452365f4d4bb68dd5e12111f10ce6]) successfully deleted
I20260926 04:04:38.245898 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.245729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41007 (local address 127.0.0.1:33380)
0926 04:04:38.245798 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:38.245845 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:41007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 04:04:38.246222 12035 catalog_manager.cc:4584] TS afba6b54a97f46d195fc37a23cc1c43c (127.0.200.129:41007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fe6fc1b77854c688253c4d828566629: Network error: Client connection negotiation failed: client connection to 127.0.200.129:41007: connect: Connection refused (error 111)
I20260926 04:04:38.246269 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 12404
I20260926 04:04:38.247816 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:04:38.247713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:43115 (local address 127.0.0.1:45232)
0926 04:04:38.247761 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:04:38.247786 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.130:43115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 04:04:38.251315 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 12017
I20260926 04:04:38.261423 802 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 04:04:38.606675 802 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 04:04:38.606814 802 mini_ranger.cc:66] Stopping Ranger...
I20260926 04:04:38.952020 802 mini_ranger.cc:68] Stopped Ranger
2026-09-26 04:04:38.952 UTC [11603] LOG: received smart shutdown request
2026-09-26 04:04:38.956 UTC [11603] LOG: background worker "logical replication launcher" (PID 11612) exited with exit code 1
2026-09-26 04:04:38.956 UTC [11607] LOG: shutting down
2026-09-26 04:04:39.174 UTC [11603] LOG: database system is shut down
2026-09-26T04:04:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49248 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:04:39 dist-test-slave-j8c1 krb5kdc[12586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:04:39 dist-test-slave-j8c1 krb5kdc[12586](info): set up 2 sockets
Sep 26 04:04:39 dist-test-slave-j8c1 krb5kdc[12586](info): commencing operation
krb5kdc: starting...
W20260926 04:04:40.322435 802 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:04:40 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:04:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangeradmin_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangeradmin_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangerlookup_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangerlookup_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
I20260926 04:04:40.493891 802 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIcgGpu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/postgres -l logfile start
2026-09-26 04:04:43.206 UTC [12616] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 04:04:43.206 UTC [12616] LOG: listening on IPv4 address "127.0.200.148", port 52341
2026-09-26 04:04:43.211 UTC [12616] LOG: listening on Unix socket "/tmp/.s.PGSQL.52341"
2026-09-26 04:04:43.246 UTC [12619] LOG: database system was shut down at 2026-09-26 04:04:41 UTC
2026-09-26 04:04:43.251 UTC [12616] LOG: database system is ready to accept connections
I20260926 04:04:44.210834 802 mini_postgres.cc:96] Postgres bound to 52341
2026-09-26 04:04:44.217 UTC [12627] FATAL: database "slave" does not exist
127.0.200.148:52341 - accepting connections
I20260926 04:04:44.217893 802 mini_ranger.cc:161] Starting Ranger...
I20260926 04:04:44.233707 802 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 04:04:44.853493 802 mini_ranger.cc:87] Created ranger Postgres database
I20260926 04:04:44.853605 802 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin
2026-09-26 04:04:45,137 [I] DB FLAVOR :POSTGRES
2026-09-26 04:04:45,138 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 04:04:45,138 [I] Checking connection..
2026-09-26 04:04:45,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 04:04:45,396 [I] Checking connection passed.
2026-09-26 04:04:45,396 [I] --------- Verifying version history table ---------
2026-09-26 04:04:45,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:04:45,669 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 04:04:45,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:04:45,903 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 04:04:45,903 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 04:04:45,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 04:04:46.168 UTC [12691] WARNING: there is no transaction in progress
2026-09-26 04:04:46,175 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 04:04:46,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 04:04:46,438 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 04:04:46,438 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 04:04:46,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 04:04:46,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 04:04:46,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j8c1.c.gcp-upstream.internal','N') ;"
2026-09-26 04:04:47,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 04:04:47,468 [I] Table x_portal_user does not exist in database ranger
2026-09-26 04:04:47,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 04:04:47,703 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 04:04:47,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 04:04:47,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 04:04:48,205 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 04:04:48,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 04:04:49.325 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.349 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.370 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.392 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.413 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.437 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.504 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.512 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.529 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.541 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.555 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.574 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.588 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.596 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.605 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:49.615 UTC [12799] WARNING: there is no transaction in progress
2026-09-26 04:04:50,491 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 04:04:50,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 04:04:50,752 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 04:04:50,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 04:04:51,031 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 04:04:51,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 04:04:51,308 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 04:04:51,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 04:04:51,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 04:04:51,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j8c1.c.gcp-upstream.internal';"
2026-09-26 04:04:52,096 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 04:04:52,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 04:04:52,347 [I] DB_PATCHES have already been applied
I20260926 04:04:52.353417 802 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIcgGpu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 04:04:52.353485 802 mini_ranger.cc:193] Using host: 127.0.200.148
I20260926 04:04:52.356554 802 mini_ranger.cc:227] Ranger admin URL: http://127.0.200.148:44047
Sep 26, 2026 4:04:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:04:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:04:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:04:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:04:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:04:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:04:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 04:04:53.483683 802 mini_ranger.cc:160] Time spent starting Ranger: real 9.266s user 0.000s sys 0.009s
Sep 26, 2026 4:05:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:05:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 4:05:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:05:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:05:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 04:05:16.917361 802 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.200.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.200.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangerkms_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.200.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/rangerkms_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.200.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.200.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.200.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/HTTP_127.0.200.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 04:05:16 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395516, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 04:05:17.005168 802 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 04:05:17.014503 802 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 04:05:18.031718 802 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 04:05:18.031804 802 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms
I20260926 04:05:18.031836 802 mini_ranger_kms.cc:227] Using postgres at 127.0.200.148:52341
2026-09-26 04:05:18,196 [I] DB FLAVOR :POSTGRES
2026-09-26 04:05:18,196 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 04:05:18,197 [I] Checking connection
2026-09-26 04:05:18,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 04:05:18,462 [I] connection success
2026-09-26 04:05:18,462 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 04:05:18,462 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 04:05:18,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 04:05:18,703 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 04:05:18,703 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 04:05:18,704 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 04:05:18,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.200.148:52341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 04:05:19,009 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 04:05:19.524726 802 mini_ranger_kms.cc:310] Created kms service
I20260926 04:05:19.753502 802 mini_ranger_kms.cc:326] Created kudu user
I20260926 04:05:19.803491 802 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 04:05:19 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395516, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:20.222059 802 mini_ranger_kms.cc:384] Added ranger policy
I20260926 04:05:20.222271 802 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIcgGpu/build/release/bin/postgresql.jar:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIcgGpu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 04:05:20.222296 802 mini_ranger_kms.cc:240] Using host: 127.0.200.148
I20260926 04:05:20.225618 802 mini_ranger_kms.cc:276] Ranger KMS PID: 12980
I20260926 04:05:20.225708 802 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.200.148:60127
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:05:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 04:05:22.518857 802 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.514s user 0.011s sys 0.136s
I20260926 04:05:22.518951 802 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 04:05:22.518996 802 mini_ranger_kms.cc:401] 127.0.200.148:60127/kms/v1/keys
Sep 26 04:05:22 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790395522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 04:05:22 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
Sep 26 04:05:23 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395522, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.200.148@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
Sep 26 04:05:25 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:05:25.691946 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46599
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:46599
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:25.773700 13030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:25.773910 13030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:25.773932 13030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:25.775157 13030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:25.775200 13030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:25.775215 13030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:25.775228 13030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:25.775239 13030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:25.776927 13030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.200.148:60127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:46599
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:25.777179 13030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:25.777420 13030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:25.783156 13035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:25.783257 13036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:25.783159 13038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:25.783282 13030 server_base.cc:1034] running on GCE node
I20260926 04:05:25.783655 13030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:25.783986 13030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:25.785199 13030 hybrid_clock.cc:648] HybridClock initialized: now 1790395525785185 us; error 33 us; skew 500 ppm
Sep 26 04:05:25 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:25.788105 13030 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:25.788663 13030 webserver.cc:460] Webserver started at http://127.0.200.190:43957/ using document root <none> and password file <none>
I20260926 04:05:25.788906 13030 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:25.788966 13030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:25.789124 13030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:05:25 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:25.837412 13030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "c5aee7552650444faea9a06f8333d813"
format_stamp: "Formatted at 2026-09-26 04:05:25 on dist-test-slave-j8c1"
server_key: "4b5a563e41c27e66e6d725ef59c92e1f"
server_key_iv: "d194870639cb6ec2b2c8555f44e85815"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:25.837847 13030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "c5aee7552650444faea9a06f8333d813"
format_stamp: "Formatted at 2026-09-26 04:05:25 on dist-test-slave-j8c1"
server_key: "4b5a563e41c27e66e6d725ef59c92e1f"
server_key_iv: "d194870639cb6ec2b2c8555f44e85815"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:25.839404 13030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:05:25.855154 13045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:25.855417 13030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:25.855526 13030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "c5aee7552650444faea9a06f8333d813"
format_stamp: "Formatted at 2026-09-26 04:05:25 on dist-test-slave-j8c1"
server_key: "4b5a563e41c27e66e6d725ef59c92e1f"
server_key_iv: "d194870639cb6ec2b2c8555f44e85815"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:25.855643 13030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:25.872437 13030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:25.874302 13030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:25.874466 13030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:25.878901 13030 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:46599
I20260926 04:05:25.878940 13097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:46599 every 8 connection(s)
I20260926 04:05:25.879280 13030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:25.879902 13098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:25.882269 13098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Bootstrap starting.
I20260926 04:05:25.882946 13098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:25.883287 13098 log.cc:824] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:25.884059 13098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: No bootstrap required, opened a new log
I20260926 04:05:25.886229 13098 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:25.886412 13098 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:25.886458 13098 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5aee7552650444faea9a06f8333d813, State: Initialized, Role: FOLLOWER
I20260926 04:05:25.886668 13098 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:25.886813 13098 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:25.886860 13098 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:25.886914 13098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:25.887522 13098 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:25.887655 13098 leader_election.cc:302] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5aee7552650444faea9a06f8333d813; no voters:
I20260926 04:05:25.887846 13098 leader_election.cc:288] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:05:25.888000 13101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:25.888154 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13030
I20260926 04:05:25.888155 13098 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:25.888247 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:05:25.888345 13101 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 LEADER]: Becoming Leader. State: Replica: c5aee7552650444faea9a06f8333d813, State: Running, Role: LEADER
I20260926 04:05:25.888414 802 external_mini_cluster.cc:1428] Setting key 61707c146be8544cccfd0fc573e30435
I20260926 04:05:25.888526 13101 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:25.888895 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5aee7552650444faea9a06f8333d813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } } }
I20260926 04:05:25.888981 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:25.889206 13103 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5aee7552650444faea9a06f8333d813. Latest consensus state: current_term: 1 leader_uuid: "c5aee7552650444faea9a06f8333d813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } } }
I20260926 04:05:25.889294 13103 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:25.889451 13112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:25.890148 13112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:25.890410 13098 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 04:05:25.895725 13112 catalog_manager.cc:1347] Generated new cluster ID: 8d91f9b0b73647589ded79228eed23db
I20260926 04:05:25.895867 13112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:05:25 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:25.904628 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:25.888960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.0.200.190:46599)
0926 04:05:25.889129 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:25.889140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:25.889860 (+ 720us) server_negotiation.cc:407] Connection header received
0926 04:05:25.890018 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:25.890026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:25.890087 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:25.890252 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:25.892311 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:25.894605 (+ 2294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:25.895821 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:25.896496 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:25.899790 (+ 3294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:25.899905 (+ 115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:25.899909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:25.899909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:25.899920 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:25.902494 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:25.902987 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:25.902991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:25.902991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:25.903097 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:25.903283 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:25.903283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:25.903283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:25.903596 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:25.904183 (+ 587us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:25.904267 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 04:05:25.904395 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":83,"threads_started":1}
I20260926 04:05:25.907068 13112 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:05:25.907637 13112 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:25.910735 13112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Generated new TSK 0
I20260926 04:05:25.910948 13112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:05:26.578754 13140 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:05:26.637940 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:26.748106 13143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:26.748361 13143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:26.748406 13143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:26.749672 13143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:26.749753 13143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:26.749802 13143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:26.751463 13143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:26.751751 13143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:26.752004 13143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:26.758610 13149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:26.758770 13143 server_base.cc:1034] running on GCE node
W20260926 04:05:26.758699 13151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:26.758617 13148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:26.759112 13143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:26.759428 13143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:26.760615 13143 hybrid_clock.cc:648] HybridClock initialized: now 1790395526760599 us; error 32 us; skew 500 ppm
Sep 26 04:05:26 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:26.763792 13143 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:26.764484 13143 webserver.cc:460] Webserver started at http://127.0.200.129:43977/ using document root <none> and password file <none>
I20260926 04:05:26.764804 13143 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:26.764927 13143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:26.765115 13143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:05:26 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:26.780038 13143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "0b482bd022d248529db082c6402d7f09"
format_stamp: "Formatted at 2026-09-26 04:05:26 on dist-test-slave-j8c1"
server_key: "2ccf07ad8571aec1cffe2a06528864f8"
server_key_iv: "264d7d55ce677b1f57bcc04cbf7593cc"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:26.780633 13143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "0b482bd022d248529db082c6402d7f09"
format_stamp: "Formatted at 2026-09-26 04:05:26 on dist-test-slave-j8c1"
server_key: "2ccf07ad8571aec1cffe2a06528864f8"
server_key_iv: "264d7d55ce677b1f57bcc04cbf7593cc"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:26.783175 13143 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 04:05:26.798278 13160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:26.798861 13143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:26.799021 13143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "0b482bd022d248529db082c6402d7f09"
format_stamp: "Formatted at 2026-09-26 04:05:26 on dist-test-slave-j8c1"
server_key: "2ccf07ad8571aec1cffe2a06528864f8"
server_key_iv: "264d7d55ce677b1f57bcc04cbf7593cc"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:26.799146 13143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:26.813700 13143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:26.816499 13143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:26.816743 13143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:26.817077 13143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:26.817483 13143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:26.817560 13143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:26.817656 13143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:26.817723 13143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:26.825271 13143 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:46541
I20260926 04:05:26.825501 13273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:46541 every 8 connection(s)
I20260926 04:05:26.825789 13143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:05:26 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:26.835467 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13143
I20260926 04:05:26.835574 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:05:26.835768 802 external_mini_cluster.cc:1428] Setting key 06e52d87af5b84ebe5d4002c78a24ed2
I20260926 04:05:26.845407 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:26.826638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.129:42135)
0926 04:05:26.826900 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:26.826942 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:26.827575 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:26.828026 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:26.828030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:26.828285 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:26.828583 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:26.828589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:26.829616 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:26.829617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:26.830907 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:26.830910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:26.831489 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:26.831490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:26.831677 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:26.832395 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:26.832406 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:26.834144 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:26.843455 (+ 9311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:26.843470 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:26.843473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:26.843833 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:26.844127 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:26.844128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:26.844128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:26.844295 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:26.844644 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:26.844652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:26.844845 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 04:05:26.844941 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 04:05:26.845130 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":81,"threads_started":1}
I20260926 04:05:26.846321 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:26.826667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:42135 (local address 127.0.200.190:46599)
0926 04:05:26.827770 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:26.827772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:26.827781 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:26.827808 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:26.827808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:26.827822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:26.827881 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:26.828702 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:26.829504 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:26.831013 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:26.831323 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:26.831387 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:26.834280 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:26.834282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:26.834282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:26.834291 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:26.843287 (+ 8996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:26.843969 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:26.843969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:26.843970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:26.844046 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:26.844394 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:26.844395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:26.844395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:26.844560 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:26.846055 (+ 1495us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:26.846104 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 04:05:26.846208 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":76,"threads_started":1}
I20260926 04:05:26.846714 13274 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:26.846822 13274 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:26.847050 13274 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:26.847608 13062 ts_manager.cc:194] Registered new tserver with Master: 0b482bd022d248529db082c6402d7f09 (127.0.200.129:46541)
I20260926 04:05:26.848583 13062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:42135
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:05:26.880745 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:26.989836 13280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:26.990082 13280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:26.990126 13280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 26 04:05:26 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790395526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 04:05:26.991915 13280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:26.991998 13280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:26.992056 13280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:26.994385 13280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:26.994741 13280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:26.995059 13280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:27.000671 13288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.000689 13287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.001113 13290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:27.001300 13280 server_base.cc:1034] running on GCE node
I20260926 04:05:27.001525 13280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:27.001849 13280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:27.003031 13280 hybrid_clock.cc:648] HybridClock initialized: now 1790395527003012 us; error 25 us; skew 500 ppm
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:27.006531 13280 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:27.007177 13280 webserver.cc:460] Webserver started at http://127.0.200.130:40017/ using document root <none> and password file <none>
I20260926 04:05:27.007509 13280 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:27.007615 13280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:27.007802 13280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:27.033422 13280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "bfbe7012a53b4063a3d575360a5da5ba"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "01af2bda5351410900228525e196233d"
server_key_iv: "6fe2697c7b38cf78a0edfbd946ec3f74"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.033906 13280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "bfbe7012a53b4063a3d575360a5da5ba"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "01af2bda5351410900228525e196233d"
server_key_iv: "6fe2697c7b38cf78a0edfbd946ec3f74"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.038524 13280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 04:05:27.053721 13297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.053970 13280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:27.054060 13280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "bfbe7012a53b4063a3d575360a5da5ba"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "01af2bda5351410900228525e196233d"
server_key_iv: "6fe2697c7b38cf78a0edfbd946ec3f74"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.054154 13280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:27.076679 13280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:27.079301 13280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:27.079454 13280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:27.079716 13280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:27.080063 13280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:27.080098 13280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.080133 13280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:27.080152 13280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.089459 13411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:37015 every 8 connection(s)
I20260926 04:05:27.089617 13280 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:37015
I20260926 04:05:27.090154 13280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:05:27.091070 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13280
I20260926 04:05:27.091197 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:05:27.091442 802 external_mini_cluster.cc:1428] Setting key 2b8501f0797b6b232a08af0fcbbc0917
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:27.107193 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.091461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44149 (local address 127.0.200.190:46599)
0926 04:05:27.091517 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:27.091520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:27.095674 (+ 4154us) server_negotiation.cc:407] Connection header received
0926 04:05:27.096389 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:27.096391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:27.096409 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:27.096493 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:27.097412 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.098243 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.099886 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.100213 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.100265 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.103446 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:27.103448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:27.103448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:27.103457 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:27.105376 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.105959 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.105961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.105961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.106022 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.106293 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.106294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.106294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.106436 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:27.106488 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:27.106951 (+ 463us) server_negotiation.cc:299] Negotiation successful
0926 04:05:27.107055 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:05:27.108058 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.091445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.130:44149)
0926 04:05:27.095439 (+ 3994us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:27.095514 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:27.096255 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:27.096656 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:27.096661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:27.096955 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:27.097278 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.097285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.098371 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.098372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.099763 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.099767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.100436 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.100437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.100595 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.101290 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:27.101301 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:27.103293 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:27.105495 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.105505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.105508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.105868 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.106087 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.106087 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.106087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.106226 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.106520 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:27.106526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:27.106742 (+ 216us) client_negotiation.cc:769] Sending connection context
0926 04:05:27.107590 (+ 848us) client_negotiation.cc:240] Negotiation successful
0926 04:05:27.107781 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3924,"thread_start_us":99,"threads_started":1}
I20260926 04:05:27.108909 13412 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:27.109010 13412 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:27.109210 13412 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:05:27.109656 13062 ts_manager.cc:194] Registered new tserver with Master: bfbe7012a53b4063a3d575360a5da5ba (127.0.200.130:37015)
I20260926 04:05:27.110172 13062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:44149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:05:27.137799 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:27.273942 13417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:27.274320 13417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:27.274410 13417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:27.276365 13417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:27.276624 13417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:27.276713 13417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:27.279088 13417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:27.279596 13417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:27.279913 13417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:27.286376 13423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.286998 13425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.287323 13422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:27.288628 13417 server_base.cc:1034] running on GCE node
I20260926 04:05:27.288926 13417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:27.289244 13417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:27.304942 13417 hybrid_clock.cc:648] HybridClock initialized: now 1790395527304924 us; error 76 us; skew 500 ppm
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:27.309612 13417 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:27.310173 13417 webserver.cc:460] Webserver started at http://127.0.200.131:40199/ using document root <none> and password file <none>
I20260926 04:05:27.310453 13417 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:27.310521 13417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:27.310679 13417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:27.324856 13417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "796dbdb5ae4ab39240a2b8984b582f99"
server_key_iv: "2cc4fc574bdae2f41915fa3c44791205"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.325322 13417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "796dbdb5ae4ab39240a2b8984b582f99"
server_key_iv: "2cc4fc574bdae2f41915fa3c44791205"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.327142 13417 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260926 04:05:27.340386 13432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.340590 13417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:27.340674 13417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "796dbdb5ae4ab39240a2b8984b582f99"
server_key_iv: "2cc4fc574bdae2f41915fa3c44791205"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.340765 13417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:27.353101 13417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:27.355482 13417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:27.355621 13417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:27.355870 13417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:27.356235 13417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:27.356278 13417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.356314 13417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:27.356331 13417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.362372 13417 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:41505
I20260926 04:05:27.364988 13545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:41505 every 8 connection(s)
I20260926 04:05:27.365366 13417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:05:27.371994 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13417
I20260926 04:05:27.372133 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:05:27.372342 802 external_mini_cluster.cc:1428] Setting key 5347979f846099b86a8892b2617205b3
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:27.378312 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.366430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:54149 (local address 127.0.200.190:46599)
0926 04:05:27.366493 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:27.366496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:27.366841 (+ 345us) server_negotiation.cc:407] Connection header received
0926 04:05:27.367532 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:27.367533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:27.367545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:27.367575 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:27.368412 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.369132 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.371035 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.371340 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.371390 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.374497 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:27.374499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:27.374499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:27.374508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:27.376424 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.377011 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.377013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.377013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.377079 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.377495 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.377496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.377497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.377646 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:27.377691 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:27.378113 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 04:05:27.378207 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 04:05:27.379194 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.366426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.131:54149)
0926 04:05:27.366717 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:27.366758 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:27.367406 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:27.367755 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:27.367759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:27.368037 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:27.368306 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.368312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.369246 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.369247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.370910 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.370914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.371508 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.371509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.371718 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.372479 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:27.372490 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:27.374345 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:27.376531 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.376541 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.376544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.376914 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.377253 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.377254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.377255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.377409 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.377741 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:27.377748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:27.377949 (+ 201us) client_negotiation.cc:769] Sending connection context
0926 04:05:27.378715 (+ 766us) client_negotiation.cc:240] Negotiation successful
0926 04:05:27.378931 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":92,"threads_started":1}
I20260926 04:05:27.379940 13546 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:27.380043 13546 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:27.380244 13546 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:27.380710 13062 ts_manager.cc:194] Registered new tserver with Master: 9ba5ac41ce8a423baa4d85dadfd3dea7 (127.0.200.131:41505)
I20260926 04:05:27.381243 13062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:54149
I20260926 04:05:27.386677 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:05:27.387331 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13143
I20260926 04:05:27.394485 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13280
I20260926 04:05:27.401870 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13417
I20260926 04:05:27.409065 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13030
I20260926 04:05:27.416368 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46599
--webserver_interface=127.0.200.190
--webserver_port=43957
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:46599
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:27.498714 13549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:27.498935 13549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:27.498963 13549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:27.500248 13549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:27.500290 13549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:27.500305 13549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:27.500319 13549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:27.500332 13549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:27.502106 13549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:46599
--ranger_config_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:46599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:27.502368 13549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:27.502650 13549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:27.507513 13554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.507565 13555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.507546 13557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:27.507704 13549 server_base.cc:1034] running on GCE node
I20260926 04:05:27.507975 13549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:27.508256 13549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:27.509428 13549 hybrid_clock.cc:648] HybridClock initialized: now 1790395527509397 us; error 46 us; skew 500 ppm
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:27.512019 13549 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:27.512499 13549 webserver.cc:460] Webserver started at http://127.0.200.190:43957/ using document root <none> and password file <none>
I20260926 04:05:27.512730 13549 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:27.512804 13549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:27.580643 13549 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:27.582369 13565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.582683 13549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 04:05:27.582809 13549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "c5aee7552650444faea9a06f8333d813"
format_stamp: "Formatted at 2026-09-26 04:05:25 on dist-test-slave-j8c1"
server_key: "4b5a563e41c27e66e6d725ef59c92e1f"
server_key_iv: "d194870639cb6ec2b2c8555f44e85815"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.583242 13549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:27.602927 13549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:27.605054 13549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:27.605264 13549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:27.609884 13549 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:46599
I20260926 04:05:27.609920 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:46599 every 8 connection(s)
I20260926 04:05:27.610332 13549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:27.611527 13618 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:05:27.612195 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Bootstrap starting.
I20260926 04:05:27.613183 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13549
I20260926 04:05:27.613484 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:46541
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:46599
--webserver_port=43977
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:27.620301 13618 log.cc:824] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:27.622128 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:27.622427 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Bootstrap complete.
I20260926 04:05:27.624826 13618 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:27.625095 13618 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5aee7552650444faea9a06f8333d813, State: Initialized, Role: FOLLOWER
I20260926 04:05:27.625280 13618 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:27.625366 13618 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:27.625638 13618 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:27.625746 13618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:05:27.626986 13618 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:27.627122 13618 leader_election.cc:302] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5aee7552650444faea9a06f8333d813; no voters:
I20260926 04:05:27.627337 13618 leader_election.cc:288] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 04:05:27.627385 13622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 04:05:27.627580 13622 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [term 2 LEADER]: Becoming Leader. State: Replica: c5aee7552650444faea9a06f8333d813, State: Running, Role: LEADER
I20260926 04:05:27.627656 13618 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:27.627681 13622 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } }
I20260926 04:05:27.627974 13622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5aee7552650444faea9a06f8333d813. Latest consensus state: current_term: 2 leader_uuid: "c5aee7552650444faea9a06f8333d813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } } }
I20260926 04:05:27.627977 13624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c5aee7552650444faea9a06f8333d813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aee7552650444faea9a06f8333d813" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 46599 } } }
I20260926 04:05:27.628057 13622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:27.628082 13624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:27.629561 13618 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 04:05:27.631868 13626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:27.633019 13626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:27.633261 13626 catalog_manager.cc:1259] Loaded cluster ID: 8d91f9b0b73647589ded79228eed23db
I20260926 04:05:27.633313 13626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:05:27.635227 13626 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:27.635439 13626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Loaded TSK: 0
I20260926 04:05:27.635676 13626 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 04:05:27.702054 13620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:27.702283 13620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:27.702322 13620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:27.703578 13620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:27.703645 13620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:27.703691 13620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:27.705240 13620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:46541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=43977
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:27.705533 13620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:27.705790 13620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:27.711411 13651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.711410 13653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.711762 13650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:27.711992 13620 server_base.cc:1034] running on GCE node
I20260926 04:05:27.712193 13620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:27.712394 13620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:27.713550 13620 hybrid_clock.cc:648] HybridClock initialized: now 1790395527713529 us; error 36 us; skew 500 ppm
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:27.716868 13620 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:27.717419 13620 webserver.cc:460] Webserver started at http://127.0.200.129:43977/ using document root <none> and password file <none>
I20260926 04:05:27.717705 13620 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:27.717792 13620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:27.739454 13620 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:27.740424 13666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.740641 13620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:27.740736 13620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "0b482bd022d248529db082c6402d7f09"
format_stamp: "Formatted at 2026-09-26 04:05:26 on dist-test-slave-j8c1"
server_key: "2ccf07ad8571aec1cffe2a06528864f8"
server_key_iv: "264d7d55ce677b1f57bcc04cbf7593cc"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.741247 13620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:27.772334 13620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:27.775000 13620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:27.775236 13620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:27.775542 13620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:27.775985 13620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:27.776072 13620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.776144 13620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:27.776199 13620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.783790 13620 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:46541
I20260926 04:05:27.784094 13779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:46541 every 8 connection(s)
I20260926 04:05:27.784389 13620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:05:27.789017 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13620
I20260926 04:05:27.789326 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:37015
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:46599
--webserver_port=40017
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:27.799240 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.785404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.129:40783)
0926 04:05:27.785682 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:27.785727 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:27.786401 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:27.787185 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:27.787189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:27.787457 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:27.787770 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.787777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.789681 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.789683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.791137 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.791140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.792049 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.792051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.792217 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.792900 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:27.792910 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:27.794835 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:27.797272 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.797278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.797281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.797594 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.797890 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:27.797891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:27.797891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:27.798031 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:27.798532 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:27.798538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:27.798719 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 04:05:27.798817 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 04:05:27.798997 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":96,"threads_started":1}
I20260926 04:05:27.800226 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.785516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:40783 (local address 127.0.200.190:46599)
0926 04:05:27.786667 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:27.786678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:27.786728 (+ 50us) server_negotiation.cc:407] Connection header received
0926 04:05:27.786884 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:27.786891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:27.786963 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:27.787934 (+ 971us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:27.788033 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.789546 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.791261 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.791919 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.792040 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:27.794998 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:27.795003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:27.795003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:27.795015 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:27.797167 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.797701 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.797704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.797705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.797815 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:27.798108 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:27.798109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:27.798109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:27.798438 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:27.799759 (+ 1321us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:27.799858 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 04:05:27.800008 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":97,"threads_started":1}
I20260926 04:05:27.801301 13780 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:27.801455 13780 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:27.801699 13780 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:27.802860 13780 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:46599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 04:05:27.803045 13780 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:27.803172 13780 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:27.803638 13780 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:27.803771 13780 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:27.804204 13780 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:05:27.910931 13784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:27.911199 13784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:27.911240 13784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:27.913048 13784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:27.913103 13784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:27.913144 13784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:27.915513 13784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:37015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=40017
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:27.915772 13784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:27.916061 13784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:27.921545 13792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.921603 13790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:27.921646 13789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:27.922006 13784 server_base.cc:1034] running on GCE node
I20260926 04:05:27.922228 13784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:27.922502 13784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:27.923709 13784 hybrid_clock.cc:648] HybridClock initialized: now 1790395527923693 us; error 27 us; skew 500 ppm
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:27.927598 13784 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:27.928155 13784 webserver.cc:460] Webserver started at http://127.0.200.130:40017/ using document root <none> and password file <none>
I20260926 04:05:27.928431 13784 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:27.928505 13784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 04:05:27 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:27.951012 13784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260926 04:05:27.953466 13800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.953701 13784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 04:05:27.953824 13784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "bfbe7012a53b4063a3d575360a5da5ba"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "01af2bda5351410900228525e196233d"
server_key_iv: "6fe2697c7b38cf78a0edfbd946ec3f74"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:27.954370 13784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:27.983844 13784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:27.986315 13784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:27.986523 13784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:27.986871 13784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:27.987282 13784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:27.987327 13784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.987377 13784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:27.987406 13784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:27.995034 13784 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:37015
I20260926 04:05:27.995510 13784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:05:27.996078 13913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:37015 every 8 connection(s)
I20260926 04:05:28.001645 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13784
I20260926 04:05:28.001948 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.200.148:60127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:41505
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:46599
--webserver_port=40199
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 04:05:28 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:28.008169 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.996477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:59451 (local address 127.0.200.190:46599)
0926 04:05:27.996531 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:27.996533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:27.996716 (+ 183us) server_negotiation.cc:407] Connection header received
0926 04:05:27.997279 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:27.997281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:27.997297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:27.997351 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:27.998060 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.998633 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.999888 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.000209 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.000285 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:28.004421 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:28.004423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:28.004424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:28.004434 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:28.006328 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.006902 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.006903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.006904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.006984 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.007316 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.007318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.007318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.007481 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:28.007561 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:28.007916 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 04:05:28.008026 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:05:28.008328 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:27.996287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.130:59451)
0926 04:05:27.996586 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:27.996631 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:27.997170 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:27.997416 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:27.997420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:27.997660 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:27.997882 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.997887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:27.998739 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:27.998741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:27.999811 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:27.999814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.000316 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.000318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:28.000452 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:28.001046 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:28.001055 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:28.004305 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:28.006436 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:28.006445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:28.006448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:28.006806 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:28.007076 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:28.007077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:28.007077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:28.007231 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:28.007565 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:28.007572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:28.007741 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 04:05:28.007880 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 04:05:28.008078 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":100,"threads_started":1}
I20260926 04:05:28.009007 13914 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:28.009123 13914 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.009379 13914 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:28.010061 13914 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:46599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 04:05:28.010174 13914 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.010281 13914 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:28.010686 13914 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.010811 13914 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:28.011183 13914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:05:28.106504 13917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:28.106894 13917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:28.106956 13917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:28.108438 13917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:28.108520 13917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:28.108609 13917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:28.110154 13917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.200.148:60127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:41505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=40199
--tserver_master_addrs=127.0.200.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:28.110423 13917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:28.110699 13917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:28.116074 13925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:28.116080 13922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:28.116592 13923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:28.116755 13917 server_base.cc:1034] running on GCE node
I20260926 04:05:28.116971 13917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:28.117265 13917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:28.118446 13917 hybrid_clock.cc:648] HybridClock initialized: now 1790395528118424 us; error 36 us; skew 500 ppm
Sep 26 04:05:28 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:28.120999 13917 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:28.121531 13917 webserver.cc:460] Webserver started at http://127.0.200.131:40199/ using document root <none> and password file <none>
I20260926 04:05:28.121784 13917 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:28.121876 13917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 04:05:28 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:28.150939 13917 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.000s
I20260926 04:05:28.151870 13934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:28.152355 13917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:28.152479 13917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7"
format_stamp: "Formatted at 2026-09-26 04:05:27 on dist-test-slave-j8c1"
server_key: "796dbdb5ae4ab39240a2b8984b582f99"
server_key_iv: "2cc4fc574bdae2f41915fa3c44791205"
server_key_version: "kuduclusterkey@0"
I20260926 04:05:28.152974 13917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:28.193219 13917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:28.200076 13917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:28.200327 13917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:28.200603 13917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:28.201495 13917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:28.201591 13917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:28.201659 13917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:28.201714 13917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:28.212850 13917 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:41505
I20260926 04:05:28.212944 14047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:41505 every 8 connection(s)
I20260926 04:05:28.213291 13917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
Sep 26 04:05:28 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:28.221748 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 13917
I20260926 04:05:28.225266 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:28.214121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:46599 (local address 127.0.200.131:46659)
0926 04:05:28.214287 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:28.214331 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:28.215924 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:28.216221 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:28.216225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:28.216450 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:28.216650 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:28.216654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.217347 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.217348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:28.218164 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:28.218166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.218589 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.218590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:28.218738 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:28.219342 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:28.219354 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:28.220966 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:28.223512 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:28.223520 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:28.223523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:28.223848 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:28.224067 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:28.224069 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:28.224069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:28.224212 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:28.224526 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:28.224531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:28.224704 (+ 173us) client_negotiation.cc:769] Sending connection context
0926 04:05:28.224811 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 04:05:28.224983 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260926 04:05:28.226042 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:28.214107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:46659 (local address 127.0.200.190:46599)
0926 04:05:28.214166 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:28.214168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:28.214433 (+ 265us) server_negotiation.cc:407] Connection header received
0926 04:05:28.216066 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:28.216068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:28.216081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:28.216136 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:28.216753 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.217251 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.218264 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.218483 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.218590 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:28.221092 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:28.221094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:28.221094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:28.221104 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:28.223398 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.223930 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.223931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.223932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.223992 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.224282 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.224283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.224283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.224437 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:28.224499 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:28.225827 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0926 04:05:28.225923 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 04:05:28.226722 14048 heartbeater.cc:347] Connected to a master server at 127.0.200.190:46599
I20260926 04:05:28.226823 14048 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.227015 14048 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:28.227697 14048 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:46599 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 04:05:28.227782 14048 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.227850 14048 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:28.228207 14048 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.228279 14048 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
W20260926 04:05:28.228609 14048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 04:05:28.231678 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:28.222288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.0.200.190:46599)
0926 04:05:28.222667 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:28.222670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:28.222677 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:05:28.222703 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:28.222704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:28.222718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:28.223577 (+ 859us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:28.223618 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.224387 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.225503 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:28.226381 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:28.226949 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:28.228420 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:28.228421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:28.228422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:28.228430 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:28.230221 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.230620 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.230621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.230621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.230674 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:28.230851 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:28.230851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:28.230852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:28.230963 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:28.231454 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:28.231492 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 04:05:28.231570 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":84,"threads_started":1}
Sep 26 04:05:28 dist-test-slave-j8c1 krb5kdc[12586](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790395528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:28.804442 13780 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:28.804623 13780 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:28.805063 13582 ts_manager.cc:194] Registered new tserver with Master: 0b482bd022d248529db082c6402d7f09 (127.0.200.129:46541)
I20260926 04:05:28.805856 13582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:40783
I20260926 04:05:29.011428 13914 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:29.011600 13914 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:29.012068 13582 ts_manager.cc:194] Registered new tserver with Master: bfbe7012a53b4063a3d575360a5da5ba (127.0.200.130:37015)
I20260926 04:05:29.012625 13582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:59451
Sep 26 04:05:29 dist-test-slave-j8c1 krb5kdc[12586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for HTTP/127.0.200.148@KRBTEST.COM
I20260926 04:05:29.228767 14048 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:29.228947 14048 heartbeater.cc:511] Master 127.0.200.190:46599 requested a full tablet report, sending...
I20260926 04:05:29.229311 13582 ts_manager.cc:194] Registered new tserver with Master: 9ba5ac41ce8a423baa4d85dadfd3dea7 (127.0.200.131:41505)
I20260926 04:05:29.229795 13582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:46659
I20260926 04:05:29.234243 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:05:29.244268 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.235424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.0.200.190:46599)
0926 04:05:29.235907 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.235909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.235917 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:29.235940 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.235941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.235953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.236836 (+ 883us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:29.236884 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.237661 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.238664 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.238945 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.239001 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.240643 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:29.240644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:29.240645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:29.240653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:29.242661 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:29.243063 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:29.243064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:29.243065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:29.243133 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:29.243361 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:29.243362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:29.243363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:29.243503 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:29.243598 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.244070 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.244157 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":66,"threads_started":1}
I20260926 04:05:29.248255 13582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:05:29.250787 13582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:05:29.267544 14075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.256439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:46541 (local address 127.0.0.1:50014)
0926 04:05:29.262640 (+ 6201us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.262654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:29.262773 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:29.263096 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:29.263100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:29.263102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:29.263185 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.263188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.264518 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.264519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.265857 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.265861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.267027 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.267028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.267257 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.267281 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 04:05:29.267380 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 04:05:29.267442 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6178,"thread_start_us":100,"threads_started":1}
I20260926 04:05:29.268170 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.257062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.0.200.129:46541)
0926 04:05:29.257207 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.257213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.262734 (+ 5521us) server_negotiation.cc:407] Connection header received
0926 04:05:29.262869 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.262872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.262891 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.262954 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:29.263272 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.264403 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.265957 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.266814 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.266893 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.266987 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.268018 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.268067 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260926 04:05:29.269366 13698 tablet_service.cc:1467] Processing CreateTablet for tablet a4b4beccbb2e40368d78c5231df48057 (DEFAULT_TABLE table=test-table [id=09682d0f6a8948e3a52649460ca28461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:29.269752 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b4beccbb2e40368d78c5231df48057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:29.277067 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.256413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:37015 (local address 127.0.0.1:39646)
0926 04:05:29.256856 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.256902 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:29.256972 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:29.268775 (+ 11803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:29.268776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:29.268779 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:29.268836 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.268837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.274671 (+ 5834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.274673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.275854 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.275857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.276831 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.276832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.276904 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.276910 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:05:29.276939 (+ 29us) client_negotiation.cc:240] Negotiation successful
0926 04:05:29.276981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":122,"threads_started":1}
I20260926 04:05:29.277725 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.258664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.0.200.130:37015)
0926 04:05:29.268583 (+ 9919us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.268589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.268602 (+ 13us) server_negotiation.cc:407] Connection header received
0926 04:05:29.268640 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.268643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.268661 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.273341 (+ 4680us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:29.273435 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.274531 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.275959 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.276734 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.277476 (+ 742us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.277574 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.277601 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.277642 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9866,"thread_start_us":54,"threads_started":1}
I20260926 04:05:29.278366 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.256686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:41505 (local address 127.0.0.1:45388)
0926 04:05:29.265999 (+ 9313us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.266010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:29.266051 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:29.274883 (+ 8832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:29.274884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:29.274887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:29.274924 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.274926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.276111 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.276112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.277133 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.277135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.278146 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.278147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.278205 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.278209 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:29.278257 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 04:05:29.278297 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9294,"thread_start_us":104,"threads_started":1}
I20260926 04:05:29.278532 13835 tablet_service.cc:1467] Processing CreateTablet for tablet a4b4beccbb2e40368d78c5231df48057 (DEFAULT_TABLE table=test-table [id=09682d0f6a8948e3a52649460ca28461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:29.278831 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.257247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.0.200.131:41505)
0926 04:05:29.274660 (+ 17413us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.274665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.274676 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:05:29.274710 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.274712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.274727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.274804 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:29.274997 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.276011 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.277232 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.277953 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.278029 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.278109 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.278703 (+ 594us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.278751 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17370,"thread_start_us":91,"threads_started":1}
I20260926 04:05:29.278924 13835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b4beccbb2e40368d78c5231df48057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:29.279428 13969 tablet_service.cc:1467] Processing CreateTablet for tablet a4b4beccbb2e40368d78c5231df48057 (DEFAULT_TABLE table=test-table [id=09682d0f6a8948e3a52649460ca28461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:29.279704 13969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b4beccbb2e40368d78c5231df48057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:29.283711 14083 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Bootstrap starting.
I20260926 04:05:29.283783 14081 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: Bootstrap starting.
I20260926 04:05:29.284451 14083 tablet_bootstrap.cc:654] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:29.284449 14081 tablet_bootstrap.cc:654] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:29.284775 14081 log.cc:824] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:29.284837 14083 log.cc:824] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:29.285210 14082 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Bootstrap starting.
I20260926 04:05:29.285614 14083 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: No bootstrap required, opened a new log
I20260926 04:05:29.285677 14083 ts_tablet_manager.cc:1397] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:29.285879 14082 tablet_bootstrap.cc:654] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:29.286202 14082 log.cc:824] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:29.288288 14083 raft_consensus.cc:348] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.288444 14083 raft_consensus.cc:374] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:29.288473 14083 raft_consensus.cc:729] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbe7012a53b4063a3d575360a5da5ba, State: Initialized, Role: FOLLOWER
I20260926 04:05:29.289171 14083 consensus_queue.cc:260] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.289530 14083 ts_tablet_manager.cc:1428] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 04:05:29.289697 13914 heartbeater.cc:503] Master 127.0.200.190:46599 was elected leader, sending a full tablet report...
I20260926 04:05:29.295594 14081 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: No bootstrap required, opened a new log
I20260926 04:05:29.295678 14081 ts_tablet_manager.cc:1397] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260926 04:05:29.298215 14081 raft_consensus.cc:348] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.298386 14081 raft_consensus.cc:374] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:29.298415 14081 raft_consensus.cc:729] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b482bd022d248529db082c6402d7f09, State: Initialized, Role: FOLLOWER
I20260926 04:05:29.299106 14081 consensus_queue.cc:260] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.299485 14081 ts_tablet_manager.cc:1428] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 04:05:29.299651 13780 heartbeater.cc:503] Master 127.0.200.190:46599 was elected leader, sending a full tablet report...
I20260926 04:05:29.300765 14082 tablet_bootstrap.cc:492] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: No bootstrap required, opened a new log
I20260926 04:05:29.300838 14082 ts_tablet_manager.cc:1397] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260926 04:05:29.303594 14082 raft_consensus.cc:348] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.303838 14082 raft_consensus.cc:374] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:29.303911 14082 raft_consensus.cc:729] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba5ac41ce8a423baa4d85dadfd3dea7, State: Initialized, Role: FOLLOWER
I20260926 04:05:29.304101 14082 consensus_queue.cc:260] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.304528 14082 ts_tablet_manager.cc:1428] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260926 04:05:29.304692 14048 heartbeater.cc:503] Master 127.0.200.190:46599 was elected leader, sending a full tablet report...
I20260926 04:05:29.325060 14087 raft_consensus.cc:482] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:29.325230 14087 raft_consensus.cc:504] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.325661 14087 leader_election.cc:288] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b482bd022d248529db082c6402d7f09 (127.0.200.129:46541), 9ba5ac41ce8a423baa4d85dadfd3dea7 (127.0.200.131:41505)
I20260926 04:05:29.330334 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.326059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60637 (local address 127.0.200.129:46541)
0926 04:05:29.326123 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.326126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.326342 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:05:29.326420 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.326421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.326433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.326488 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:29.326757 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.327563 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.329288 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.329892 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.329945 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.329954 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.330210 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.330256 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 04:05:29.330334 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.326139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:60905 (local address 127.0.200.131:41505)
0926 04:05:29.326181 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.326182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.326504 (+ 322us) server_negotiation.cc:407] Connection header received
0926 04:05:29.326658 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.326659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.326669 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.326700 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:29.326888 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.327628 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.329281 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.329892 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.329945 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.329954 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.330210 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.330254 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:05:29.330627 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.325807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:46541 (local address 127.0.200.130:60637)
0926 04:05:29.326241 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.326253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:29.326359 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:29.326542 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:29.326568 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:29.326570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:29.326671 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.326673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.327700 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.327701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.329164 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.329168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.330037 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.330038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.330120 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.330127 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 04:05:29.330435 (+ 308us) client_negotiation.cc:240] Negotiation successful
0926 04:05:29.330493 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":37,"thread_start_us":130,"threads_started":1}
I20260926 04:05:29.331156 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.325877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:41505 (local address 127.0.200.130:60905)
0926 04:05:29.326453 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.326462 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:29.326574 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:29.326765 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:29.326766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:29.326767 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:29.326817 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.326818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.327868 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.327869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.329178 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:29.329180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.330057 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.330058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:29.330120 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.330124 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:29.330989 (+ 865us) client_negotiation.cc:240] Negotiation successful
0926 04:05:29.331048 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":86,"thread_start_us":63,"threads_started":1}
I20260926 04:05:29.331111 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b4beccbb2e40368d78c5231df48057" candidate_uuid: "bfbe7012a53b4063a3d575360a5da5ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b482bd022d248529db082c6402d7f09" is_pre_election: true
I20260926 04:05:29.331297 13734 raft_consensus.cc:2454] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbe7012a53b4063a3d575360a5da5ba in term 0.
I20260926 04:05:29.331671 13801 leader_election.cc:302] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b482bd022d248529db082c6402d7f09, bfbe7012a53b4063a3d575360a5da5ba; no voters:
I20260926 04:05:29.331760 13993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b4beccbb2e40368d78c5231df48057" candidate_uuid: "bfbe7012a53b4063a3d575360a5da5ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" is_pre_election: true
I20260926 04:05:29.331838 14087 raft_consensus.cc:2781] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:05:29.331884 14087 raft_consensus.cc:482] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:05:29.331916 14087 raft_consensus.cc:3037] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:29.331933 13993 raft_consensus.cc:2454] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfbe7012a53b4063a3d575360a5da5ba in term 0.
I20260926 04:05:29.333187 14087 raft_consensus.cc:504] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.333348 14087 leader_election.cc:288] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [CANDIDATE]: Term 1 election: Requested vote from peers 0b482bd022d248529db082c6402d7f09 (127.0.200.129:46541), 9ba5ac41ce8a423baa4d85dadfd3dea7 (127.0.200.131:41505)
I20260926 04:05:29.333489 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b4beccbb2e40368d78c5231df48057" candidate_uuid: "bfbe7012a53b4063a3d575360a5da5ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b482bd022d248529db082c6402d7f09"
I20260926 04:05:29.333580 13734 raft_consensus.cc:3037] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:29.333674 13993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b4beccbb2e40368d78c5231df48057" candidate_uuid: "bfbe7012a53b4063a3d575360a5da5ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7"
I20260926 04:05:29.333755 13993 raft_consensus.cc:3037] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:29.334621 13734 raft_consensus.cc:2454] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbe7012a53b4063a3d575360a5da5ba in term 1.
I20260926 04:05:29.334681 13993 raft_consensus.cc:2454] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfbe7012a53b4063a3d575360a5da5ba in term 1.
I20260926 04:05:29.334872 13804 leader_election.cc:302] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ba5ac41ce8a423baa4d85dadfd3dea7, bfbe7012a53b4063a3d575360a5da5ba; no voters:
I20260926 04:05:29.334947 14087 raft_consensus.cc:2781] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:29.335098 14087 raft_consensus.cc:686] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 1 LEADER]: Becoming Leader. State: Replica: bfbe7012a53b4063a3d575360a5da5ba, State: Running, Role: LEADER
I20260926 04:05:29.335209 14087 consensus_queue.cc:237] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } }
I20260926 04:05:29.336331 13582 catalog_manager.cc:5495] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba reported cstate change: term changed from 0 to 1, leader changed from <none> to bfbe7012a53b4063a3d575360a5da5ba (127.0.200.130). New cstate: current_term: 1 leader_uuid: "bfbe7012a53b4063a3d575360a5da5ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfbe7012a53b4063a3d575360a5da5ba" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 37015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 } health_report { overall_health: UNKNOWN } } }
I20260926 04:05:29.360538 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.357132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.0.200.130:37015)
0926 04:05:29.357187 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:29.357189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:29.357198 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:29.357246 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:29.357247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:29.357287 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:29.357359 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:05:29.357557 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.358406 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.359541 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:29.359832 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:29.359909 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:29.360147 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:05:29.360280 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:05:29.360372 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:29.360434 (+ 62us) server_negotiation.cc:299] Negotiation successful
0926 04:05:29.360458 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 04:05:29.362653 13734 raft_consensus.cc:1261] T a4b4beccbb2e40368d78c5231df48057 P 0b482bd022d248529db082c6402d7f09 [term 1 FOLLOWER]: Refusing update from remote peer bfbe7012a53b4063a3d575360a5da5ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:29.362653 13993 raft_consensus.cc:1261] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 1 FOLLOWER]: Refusing update from remote peer bfbe7012a53b4063a3d575360a5da5ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:29.363015 14087 consensus_queue.cc:1035] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ba5ac41ce8a423baa4d85dadfd3dea7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 41505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:29.363193 14092 consensus_queue.cc:1035] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b482bd022d248529db082c6402d7f09" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:29.368325 14095 mvcc.cc:205] Tried to move back new op lower bound from 7333460088266145792 to 7333460088157384704. Current Snapshot: MvccSnapshot[applied={T|T < 7333460088266145792}]
I20260926 04:05:29.371140 14097 mvcc.cc:205] Tried to move back new op lower bound from 7333460088266145792 to 7333460088157384704. Current Snapshot: MvccSnapshot[applied={T|T < 7333460088266145792}]
I20260926 04:05:29.371280 13581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:29.371371 13581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:29.372540 13581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5aee7552650444faea9a06f8333d813: Sending DeleteTablet for 3 replicas of tablet a4b4beccbb2e40368d78c5231df48057
I20260926 04:05:29.372920 13698 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b4beccbb2e40368d78c5231df48057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:29 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:50014
I20260926 04:05:29.372963 13969 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b4beccbb2e40368d78c5231df48057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:29 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:45388
I20260926 04:05:29.373062 13835 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b4beccbb2e40368d78c5231df48057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:29 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:39646
I20260926 04:05:29.373616 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13620
I20260926 04:05:29.378731 14106 tablet_replica.cc:330] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: stopping tablet replica
I20260926 04:05:29.378954 14106 raft_consensus.cc:2229] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:29.379205 14106 raft_consensus.cc:2258] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:29.382678 14107 tablet_replica.cc:330] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: stopping tablet replica
I20260926 04:05:29.382877 14107 raft_consensus.cc:2229] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:05:29.383042 14107 raft_consensus.cc:2258] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 04:05:29.383526 13566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:46541 (error 108)
I20260926 04:05:29.383574 14107 ts_tablet_manager.cc:1905] T a4b4beccbb2e40368d78c5231df48057 P bfbe7012a53b4063a3d575360a5da5ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:29.384141 14106 ts_tablet_manager.cc:1905] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:29.385941 14106 ts_tablet_manager.cc:1918] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:29.386101 14106 log.cc:1196] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/a4b4beccbb2e40368d78c5231df48057
I20260926 04:05:29.386451 14106 ts_tablet_manager.cc:1939] T a4b4beccbb2e40368d78c5231df48057 P 9ba5ac41ce8a423baa4d85dadfd3dea7: Deleting consensus metadata
I20260926 04:05:29.387071 13569 catalog_manager.cc:4841] TS 9ba5ac41ce8a423baa4d85dadfd3dea7 (127.0.200.131:41505): tablet a4b4beccbb2e40368d78c5231df48057 (table test-table [id=09682d0f6a8948e3a52649460ca28461]) successfully deleted
I20260926 04:05:29.387962 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.387821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:46541 (local address 127.0.0.1:50022)
0926 04:05:29.387871 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.387925 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:46541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 04:05:29.387962 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13784
W20260926 04:05:29.388615 13566 catalog_manager.cc:4584] TS 0b482bd022d248529db082c6402d7f09 (127.0.200.129:46541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4b4beccbb2e40368d78c5231df48057: Network error: Client connection negotiation failed: client connection to 127.0.200.129:46541: connect: Connection refused (error 111)
I20260926 04:05:29.396560 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:29.396440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:37015 (local address 127.0.0.1:39672)
0926 04:05:29.396490 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:29.396516 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.130:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 04:05:29.396601 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13917
I20260926 04:05:29.402525 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 13549
I20260926 04:05:29.409154 802 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 04:05:29.744076 802 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 04:05:29.744164 802 mini_ranger.cc:66] Stopping Ranger...
I20260926 04:05:30.089408 802 mini_ranger.cc:68] Stopped Ranger
2026-09-26 04:05:30.089 UTC [12616] LOG: received smart shutdown request
2026-09-26 04:05:30.094 UTC [12616] LOG: background worker "logical replication launcher" (PID 12625) exited with exit code 1
2026-09-26 04:05:30.094 UTC [12620] LOG: shutting down
2026-09-26 04:05:30.307 UTC [12616] LOG: database system is shut down
2026-09-26T04:05:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51134 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:05:30 dist-test-slave-j8c1 krb5kdc[14117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:05:30 dist-test-slave-j8c1 krb5kdc[14117](info): set up 2 sockets
Sep 26 04:05:30 dist-test-slave-j8c1 krb5kdc[14117](info): commencing operation
krb5kdc: starting...
W20260926 04:05:31.466572 802 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:05:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:05:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:05:31.590930 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37333
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:31.674602 14130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:31.674878 14130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:31.674922 14130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:31.676155 14130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:31.676229 14130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:31.676256 14130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:31.676290 14130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:31.676314 14130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:31.677951 14130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:31.678253 14130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:31.678511 14130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:31.683531 14136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:31.683521 14135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:31.683583 14138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:31.683966 14130 server_base.cc:1034] running on GCE node
I20260926 04:05:31.684264 14130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:31.684573 14130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:31.685750 14130 hybrid_clock.cc:648] HybridClock initialized: now 1790395531685705 us; error 69 us; skew 500 ppm
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:31.688489 14130 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:31.689028 14130 webserver.cc:460] Webserver started at http://127.0.200.190:39437/ using document root <none> and password file <none>
I20260926 04:05:31.689286 14130 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:31.689358 14130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:31.689512 14130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:31.690673 14130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "97e6de8508a343c39c734fd4259c9fe3"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "e9fe9bb9d90822dfc82d42400dc2faa0"
server_key_iv: "547ed7babdddd59476042d1aa540d396"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.691057 14130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "97e6de8508a343c39c734fd4259c9fe3"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "e9fe9bb9d90822dfc82d42400dc2faa0"
server_key_iv: "547ed7babdddd59476042d1aa540d396"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.692544 14130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:05:31.693446 14145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:31.693678 14130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:31.693790 14130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "97e6de8508a343c39c734fd4259c9fe3"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "e9fe9bb9d90822dfc82d42400dc2faa0"
server_key_iv: "547ed7babdddd59476042d1aa540d396"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.693904 14130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:31.720424 14130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:31.722795 14130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:31.722977 14130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:31.727212 14130 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:37333
I20260926 04:05:31.727252 14197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:37333 every 8 connection(s)
I20260926 04:05:31.727653 14130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:31.728596 14198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:31.730284 14198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:05:31.736657 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14130
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:31.736755 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:05:31.736965 802 external_mini_cluster.cc:1428] Setting key c3d4b193f32208f5e207686a27e8d08a
I20260926 04:05:31.740653 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.731042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37333 (local address 127.0.0.1:36796)
0926 04:05:31.731229 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.731269 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:31.731739 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:31.732049 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:31.732051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:31.732307 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:31.732463 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.732467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.733480 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.733481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.734196 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.734198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.734936 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.734937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.735007 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.735631 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:31.735639 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:31.737151 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:31.739099 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.739105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.739108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.739312 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.739579 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.739580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.739580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.739666 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.740048 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:31.740053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:31.740117 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 04:05:31.740225 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 04:05:31.740463 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":92,"thread_start_us":81,"threads_started":1}
I20260926 04:05:31.740661 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.731058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.0.200.190:37333)
0926 04:05:31.731256 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:31.731263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:31.731359 (+ 96us) server_negotiation.cc:407] Connection header received
0926 04:05:31.731918 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:31.731921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:31.731938 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:31.732022 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:31.732550 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.733385 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.734264 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.734826 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.734916 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.737255 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:31.737262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:31.737263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:31.737270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:31.738994 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.739417 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.739420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.739420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.739503 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.739740 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.739741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.739742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.739967 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:31.740057 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:31.740250 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 04:05:31.740398 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
I20260926 04:05:31.741852 14198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:05:31.742218 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.742093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40604)
0926 04:05:31.742144 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.742188 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260926 04:05:31.742347 14148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
I20260926 04:05:31.742770 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.742680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40614)
0926 04:05:31.742724 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.742748 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 04:05:31.742877 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:34689: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.200.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
I20260926 04:05:31.761567 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:34689
--webserver_interface=127.0.200.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:31.792121 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } attempt: 1
I20260926 04:05:31.792639 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.792484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40620)
0926 04:05:31.792564 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.792609 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 04:05:31.793004 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.792905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40634)
0926 04:05:31.792952 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.792978 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 04:05:31.793087 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:34689: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
W20260926 04:05:31.842738 14205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:31.842941 14205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:31.842963 14205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:31.844135 14205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:31.844182 14205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:31.844197 14205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:31.844208 14205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:31.844264 14205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:31.845817 14205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:34689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.200.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:31.846063 14205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:31.846309 14205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:31.851140 14213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:31.851154 14211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:31.851121 14210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:31.851329 14205 server_base.cc:1034] running on GCE node
I20260926 04:05:31.851636 14205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:31.851925 14205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:31.853096 14205 hybrid_clock.cc:648] HybridClock initialized: now 1790395531853062 us; error 70 us; skew 500 ppm
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:31.855696 14205 init.cc:377] Logged in from keytab as kudu/127.0.200.189@KRBTEST.COM (short username kudu)
I20260926 04:05:31.856137 14205 webserver.cc:460] Webserver started at http://127.0.200.189:37919/ using document root <none> and password file <none>
I20260926 04:05:31.856359 14205 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:31.856405 14205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:31.856570 14205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:31.857626 14205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data/instance:
uuid: "0d3e8f82c6764cde867ba166fb925efe"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "cda252f0b08febb2643a46fda31828db"
server_key_iv: "840ec896ab6767749a698167a1bd60d5"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.857982 14205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal/instance:
uuid: "0d3e8f82c6764cde867ba166fb925efe"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "cda252f0b08febb2643a46fda31828db"
server_key_iv: "840ec896ab6767749a698167a1bd60d5"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.859354 14205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:05:31.860061 14220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:31.860240 14205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:05:31.860317 14205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "0d3e8f82c6764cde867ba166fb925efe"
format_stamp: "Formatted at 2026-09-26 04:05:31 on dist-test-slave-j8c1"
server_key: "cda252f0b08febb2643a46fda31828db"
server_key_iv: "840ec896ab6767749a698167a1bd60d5"
server_key_version: "encryptionkey@0"
I20260926 04:05:31.860368 14205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:31.861222 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } attempt: 2
I20260926 04:05:31.861608 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.861480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40636)
0926 04:05:31.861538 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.861569 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 04:05:31.861997 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.861918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40644)
0926 04:05:31.861959 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.861982 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 04:05:31.862102 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:34689: Network error: Client connection negotiation failed: client connection to 127.0.200.189:34689: connect: Connection refused (error 111)
I20260926 04:05:31.900914 14205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:31.903290 14205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:31.903409 14205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:31.907330 14205 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.189:34689
I20260926 04:05:31.907401 14272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.189:34689 every 8 connection(s)
I20260926 04:05:31.907764 14205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
I20260926 04:05:31.908854 14273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:31.910567 14273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:31.916874 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14205
I20260926 04:05:31.917025 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-1/wal/instance
I20260926 04:05:31.917354 802 external_mini_cluster.cc:1428] Setting key e78878da9aa5c1984e106cd7893202f1
I20260926 04:05:31.920832 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.911424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.0.200.190:37333)
0926 04:05:31.911509 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:31.911511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:31.911798 (+ 287us) server_negotiation.cc:407] Connection header received
0926 04:05:31.912321 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:31.912322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:31.912338 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:31.912436 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:31.913068 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.913581 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.914620 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.914827 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.914906 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.917367 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:31.917368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:31.917368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:31.917379 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:31.919181 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.919668 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.919670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.919670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.919739 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.920071 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.920071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.920072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.920226 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:31.920305 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:31.920645 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 04:05:31.920732 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 04:05:31.921061 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.911330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37333 (local address 127.0.0.1:36802)
0926 04:05:31.911631 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.911676 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:31.912213 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:31.912479 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:31.912489 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:31.912759 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:31.912972 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.912975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.913681 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.913682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.914484 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.914486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.914919 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.914921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.915050 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.915616 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:31.915625 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:31.917257 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:31.919315 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.919323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.919325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.919583 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.919858 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.919859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.919860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.919982 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.920300 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:31.920309 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:31.920497 (+ 188us) client_negotiation.cc:769] Sending connection context
0926 04:05:31.920622 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 04:05:31.920807 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
I20260926 04:05:31.921767 14273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.200.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.188@KRBTEST.COM" created.
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:05:31.932081 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.922169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.0.200.189:34689)
0926 04:05:31.922342 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:31.922346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:31.922368 (+ 22us) server_negotiation.cc:407] Connection header received
0926 04:05:31.922390 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:31.922392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:31.922405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:31.922481 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:31.922810 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.923513 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.924210 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.924511 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.924587 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.928787 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:31.928801 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:31.928801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:31.928810 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:31.930672 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.931104 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.931108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.931108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.931172 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.931444 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.931445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.931446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.931623 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:31.931718 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:31.931828 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 04:05:31.931943 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260926 04:05:31.932087 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.922081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40660)
0926 04:05:31.922147 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.922159 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:31.922201 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:31.922495 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:31.922496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:31.922670 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:31.922726 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.922728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.923617 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.923618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.924113 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.924114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.924593 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.924594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.924657 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.925144 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:31.925149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:31.928679 (+ 3530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:31.930767 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.930768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.930769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.930993 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.931242 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.931242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.931243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.931302 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.931694 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:31.931694 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:31.931734 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 04:05:31.931800 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 04:05:31.931997 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":102}
I20260926 04:05:31.932747 14273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:05:31.933138 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.933020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32822)
0926 04:05:31.933071 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.933114 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 04:05:31.933312 14224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
I20260926 04:05:31.933907 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.933767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32832)
0926 04:05:31.933835 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.933862 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 04:05:31.934046 14273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
I20260926 04:05:31.946866 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:43963
--webserver_interface=127.0.200.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:31.953277 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } attempt: 3
Sep 26 04:05:31 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:05:31.961063 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.953692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40666)
0926 04:05:31.953770 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.953785 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:31.953844 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:31.954000 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:31.954001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:31.954174 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:31.954239 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.954241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.955095 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.955096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.955624 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:31.955627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.956023 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.956024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:31.956105 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.956688 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:31.956693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:31.957959 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:31.959898 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.959899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.959900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.960102 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.960289 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:31.960289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:31.960290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:31.960339 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:31.960773 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:31.960774 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:31.960831 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 04:05:31.960898 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 04:05:31.960977 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 04:05:31.961144 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.953764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.0.200.189:34689)
0926 04:05:31.953836 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:31.953838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:31.953850 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:05:31.953907 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:31.953908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:31.953922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:31.953994 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:31.954387 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.955001 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.955723 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:31.955946 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:31.956025 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:31.958047 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:31.958049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:31.958050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:31.958057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:31.959815 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.960167 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.960168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.960168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.960229 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:31.960506 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:31.960508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:31.960508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:31.960688 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:31.960777 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:31.960956 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 04:05:31.961043 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:05:31.961477 14198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:05:31.961905 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.961783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32842)
0926 04:05:31.961837 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.961874 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 04:05:31.962309 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.962207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32854)
0926 04:05:31.962252 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.962278 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 04:05:31.962432 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
I20260926 04:05:31.983292 14273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 1
I20260926 04:05:31.983824 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.983679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32870)
0926 04:05:31.983744 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.983790 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 04:05:31.984227 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.984141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32874)
0926 04:05:31.984183 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.984209 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 04:05:31.984320 14273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
I20260926 04:05:31.993599 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 1
I20260926 04:05:31.994092 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.993964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32890)
0926 04:05:31.994026 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.994066 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 04:05:31.994498 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:31.994404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32902)
0926 04:05:31.994453 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:31.994479 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 04:05:31.994622 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
W20260926 04:05:32.032217 14280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:32.032419 14280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:32.032447 14280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:32.033592 14280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:32.033638 14280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:32.033653 14280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:32.033665 14280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:32.033677 14280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:32.035290 14280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37333,127.0.200.189:34689,127.0.200.188:43963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:43963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.200.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:32.035552 14280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:32.035810 14280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:32.040755 14286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:32.040799 14288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:32.040868 14285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:32.041138 14280 server_base.cc:1034] running on GCE node
I20260926 04:05:32.041409 14280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:32.041698 14280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:32.042870 14280 hybrid_clock.cc:648] HybridClock initialized: now 1790395532042844 us; error 45 us; skew 500 ppm
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:32.045503 14280 init.cc:377] Logged in from keytab as kudu/127.0.200.188@KRBTEST.COM (short username kudu)
I20260926 04:05:32.045995 14280 webserver.cc:460] Webserver started at http://127.0.200.188:45943/ using document root <none> and password file <none>
I20260926 04:05:32.046239 14280 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:32.046310 14280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:32.046501 14280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:32.047612 14280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data/instance:
uuid: "44213ef9a3034a3691792c39f165bd16"
format_stamp: "Formatted at 2026-09-26 04:05:32 on dist-test-slave-j8c1"
server_key: "b3fd5b023e57b353eefb12152bac6089"
server_key_iv: "40cdfe40ac74109fb33b4b83afecf5eb"
server_key_version: "encryptionkey@0"
I20260926 04:05:32.047971 14280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal/instance:
uuid: "44213ef9a3034a3691792c39f165bd16"
format_stamp: "Formatted at 2026-09-26 04:05:32 on dist-test-slave-j8c1"
server_key: "b3fd5b023e57b353eefb12152bac6089"
server_key_iv: "40cdfe40ac74109fb33b4b83afecf5eb"
server_key_version: "encryptionkey@0"
I20260926 04:05:32.049302 14280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:05:32.050092 14295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:32.050289 14280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:32.050405 14280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "44213ef9a3034a3691792c39f165bd16"
format_stamp: "Formatted at 2026-09-26 04:05:32 on dist-test-slave-j8c1"
server_key: "b3fd5b023e57b353eefb12152bac6089"
server_key_iv: "40cdfe40ac74109fb33b4b83afecf5eb"
server_key_version: "encryptionkey@0"
I20260926 04:05:32.050501 14280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:32.052466 14273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 2
I20260926 04:05:32.052937 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.052780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32906)
0926 04:05:32.052847 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.052891 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 04:05:32.053388 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.053278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32914)
0926 04:05:32.053334 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.053365 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 04:05:32.053510 14273 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
I20260926 04:05:32.066186 14280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:32.068300 14280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:32.068478 14280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:32.069789 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 2
I20260926 04:05:32.070255 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.070090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32918)
0926 04:05:32.070162 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.070207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 04:05:32.070688 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.070542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32922)
0926 04:05:32.070619 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.070653 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260926 04:05:32.070793 14198 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:43963: Network error: Client connection negotiation failed: client connection to 127.0.200.188:43963: connect: Connection refused (error 111)
I20260926 04:05:32.072889 14280 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.188:43963
I20260926 04:05:32.072917 14347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.188:43963 every 8 connection(s)
I20260926 04:05:32.073297 14280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
I20260926 04:05:32.074260 14348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:32.075894 14348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:32.082031 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14280
I20260926 04:05:32.082170 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790395371830964-802-0/minicluster-data/master-2/wal/instance
I20260926 04:05:32.082420 802 external_mini_cluster.cc:1428] Setting key 99d77128147d9979c4d1383f01864aa3
I20260926 04:05:32.086247 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.076740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36806 (local address 127.0.200.190:37333)
0926 04:05:32.076823 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.076825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.077058 (+ 233us) server_negotiation.cc:407] Connection header received
0926 04:05:32.077523 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.077524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.077535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.077619 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.078258 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.078891 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.079912 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.080101 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.080171 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.082497 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.082499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.082499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.082508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.084708 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.085198 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.085199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.085199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.085255 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.085591 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.085592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.085592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.085731 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.085801 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.086091 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.086179 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 04:05:32.086511 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.076716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:37333 (local address 127.0.0.1:36806)
0926 04:05:32.076894 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.076952 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:32.077417 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:32.077691 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:32.077694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:32.077966 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:32.078159 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.078163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.079018 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.079019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.079824 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.079827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.080208 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.080209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.080325 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.080836 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:32.080844 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:32.082395 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:32.084785 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.084792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.084794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.085116 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.085332 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.085333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.085333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.085481 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.085801 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:32.085806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:32.085928 (+ 122us) client_negotiation.cc:769] Sending connection context
0926 04:05:32.086063 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 04:05:32.086240 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260926 04:05:32.087132 14348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:05:32.092505 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.082875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.0.200.188:43963)
0926 04:05:32.083044 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.083050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.083077 (+ 27us) server_negotiation.cc:407] Connection header received
0926 04:05:32.083108 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.083110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.083125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.083197 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.083778 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.084487 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.085489 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.085959 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.086013 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.088679 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.088688 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.088688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.088695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.090707 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.091635 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.091639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.091640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.091707 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.091916 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.091917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.091917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.092079 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.092259 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.092303 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.092382 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260926 04:05:32.095079 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.087497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.0.200.189:34689)
0926 04:05:32.087554 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.087556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.087564 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:32.087586 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.087587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.087596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.087667 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.087924 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.088436 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.089359 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.089710 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.089786 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.091670 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.091671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.091672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.091680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.093795 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.094185 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.094186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.094186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.094239 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.094475 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.094476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.094476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.094682 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.094757 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.094913 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.094985 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:05:32.095108 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.087402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:34689 (local address 127.0.0.1:40670)
0926 04:05:32.087452 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.087462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:32.087500 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:32.087668 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:32.087669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:32.087805 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:32.087847 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.087848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.088510 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.088511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.089270 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.089273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.089787 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.089788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.089868 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.090193 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:32.090198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:32.091564 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:32.093875 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.093876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.093877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.094104 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.094330 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.094331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.094332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.094393 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.094759 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:32.094760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:32.094802 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 04:05:32.094887 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:05:32.094983 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 04:05:32.095434 14348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:05:32.101797 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.093461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.0.200.190:37333)
0926 04:05:32.093530 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.093532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.093540 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:32.093564 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.093565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.093575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.093629 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.094253 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.094818 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.095825 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.096159 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.096225 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.098959 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.098960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.098960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.098968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.100493 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.100833 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.100834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.100834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.100886 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.101148 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.101149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.101149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.101319 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.101550 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.101592 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.101672 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:05:32.103214 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.095711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32948)
0926 04:05:32.095759 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.095769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:32.095800 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:32.095926 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:32.095927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:32.096034 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:32.096063 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.096064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.096919 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.096921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.097821 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.097823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.098347 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.098347 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.098389 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.098969 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:32.098973 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:32.100121 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:32.102029 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.102031 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.102031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.102246 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.102449 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.102450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.102450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.102509 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.102838 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:32.102839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:32.102898 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 04:05:32.102984 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 04:05:32.103107 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 04:05:32.103349 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.095776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.0.200.188:43963)
0926 04:05:32.095810 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.095812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.095820 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:32.095861 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.095862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.095871 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.095922 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.096109 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.096820 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.097898 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.098239 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.098319 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.100227 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.100228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.100228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.100242 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.101946 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.102299 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.102300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.102300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.102366 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.102616 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.102617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.102618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.102765 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.102832 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.103040 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.103202 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260926 04:05:32.105206 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: Bootstrap starting.
I20260926 04:05:32.106102 14348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:32.106417 14348 log.cc:824] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: Log is configured to *not* fsync() on all Append() calls
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:05:32.107328 14348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: No bootstrap required, opened a new log
I20260926 04:05:32.109786 14348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.109946 14348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:32.109987 14348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44213ef9a3034a3691792c39f165bd16, State: Initialized, Role: FOLLOWER
I20260926 04:05:32.110147 14348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.110471 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.110615 14348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:32.110651 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:05:32.110980 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.102522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.0.200.189:34689)
0926 04:05:32.102588 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.102590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.102599 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:32.102645 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.102646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.102657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.102722 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.103327 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.104036 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.104793 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.105037 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.105145 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.107704 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.107706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.107706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.107714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.109591 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.109957 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.109958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.109959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.110028 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.110308 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.110309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.110310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.110466 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.110663 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.110845 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.110916 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:05:32.112639 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 04:05:32.112730 14365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:05:32.112793 14365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:05:32.112953 14280 master_runner.cc:186] Error getting master registration for 127.0.200.190:37333: OK, instance_id { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" instance_seqno: 1790395531725924 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:05:32.115552 14280 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:05:32.122442 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.114123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.0.200.190:37333)
0926 04:05:32.114172 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.114175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.114689 (+ 514us) server_negotiation.cc:407] Connection header received
0926 04:05:32.114789 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.114791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.114801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.114831 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.115546 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.116086 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.116780 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.116985 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.117076 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.118394 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.118396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.118397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.118405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.120386 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.120778 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.120779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.120780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.120846 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.121762 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.121763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.121763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.121909 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.122032 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.122235 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.122341 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:05:32.123440 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.114292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.0.200.189:34689)
0926 04:05:32.114333 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.114336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.114691 (+ 355us) server_negotiation.cc:407] Connection header received
0926 04:05:32.114795 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.114796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.114806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.114837 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.115643 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.116437 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.117554 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.117878 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.117922 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.119442 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.119444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.119444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.119452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.121648 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.122060 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.122061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.122062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.122128 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.122725 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.122727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.122727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.122878 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.122962 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.123245 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.123333 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:05:32.126686 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.114476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.0.200.188:43963)
0926 04:05:32.114522 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.114525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.118965 (+ 4440us) server_negotiation.cc:407] Connection header received
0926 04:05:32.119091 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.119092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.119103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.119137 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.119957 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.120416 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.121666 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.122024 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.122091 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.123878 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.123880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.123880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.123888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.125419 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.125784 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.125785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.125785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.125850 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.126099 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.126100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.126100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.126284 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.126400 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.126472 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.126585 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:05:32.144672 14273 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 3
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:05:32.151777 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.145105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32964 (local address 127.0.200.188:43963)
0926 04:05:32.145172 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.145174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.145183 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:32.145218 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.145219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.145228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.145297 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.145621 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.146098 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.146814 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.146997 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.147099 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.148985 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.148988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.148988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.148995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.150641 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.150976 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.150977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.150977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.151038 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.151238 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.151239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.151239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.151363 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.151439 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.151603 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.151674 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 04:05:32.151777 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.145003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32964)
0926 04:05:32.145078 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.145096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:32.145156 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:32.145303 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:32.145304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:32.145485 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:32.145530 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.145532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.146187 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.146188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.146727 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.146729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.147098 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.147099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.147150 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.147654 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:32.147658 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:32.148878 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:32.150715 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.150717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.150717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.150905 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.151113 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.151114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.151114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.151176 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.151431 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:32.151432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:32.151476 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 04:05:32.151565 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 04:05:32.151673 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 04:05:32.153108 14273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: Bootstrap starting.
I20260926 04:05:32.153779 14273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:32.154124 14273 log.cc:824] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:32.154917 14273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: No bootstrap required, opened a new log
I20260926 04:05:32.156635 14273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.156795 14273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:32.156845 14273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3e8f82c6764cde867ba166fb925efe, State: Initialized, Role: FOLLOWER
I20260926 04:05:32.157083 14273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.157439 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.157512 14273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:32.157528 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: This master's current role is: FOLLOWER
W20260926 04:05:32.159721 14386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:05:32.159793 14386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:05:32.159876 14205 master_runner.cc:186] Error getting master registration for 127.0.200.190:37333: OK, instance_id { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" instance_seqno: 1790395531725924 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:05:32.161985 14205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:05:32.169970 14198 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } attempt: 3
Sep 26 04:05:32 dist-test-slave-j8c1 krb5kdc[14117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:05:32.177417 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.170380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:43963 (local address 127.0.0.1:32976)
0926 04:05:32.170479 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:32.170493 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:32.170534 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:32.170859 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:32.170861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:32.171098 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:32.171154 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.171156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.171883 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.171884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.172368 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:32.172370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.172740 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.172741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:32.172805 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.173307 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:32.173312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:32.174737 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:32.176359 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.176360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.176361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.176539 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.176742 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:32.176743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:32.176743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:32.176801 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:32.177062 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:32.177073 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:32.177119 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:05:32.177207 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:05:32.177313 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 04:05:32.177417 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:32.170516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.0.200.188:43963)
0926 04:05:32.170659 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:32.170661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:32.170668 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:05:32.170690 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:32.170691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:32.170704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:32.170751 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:32.171242 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.171777 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.172462 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:32.172660 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:32.172742 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:32.174845 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:32.174847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:32.174847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:32.174854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:32.176255 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.176612 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.176613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.176614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.176674 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:32.176873 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:32.176874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:32.176874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:32.176991 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:32.177071 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:32.177231 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 04:05:32.177313 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260926 04:05:32.179186 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: Bootstrap starting.
I20260926 04:05:32.179822 14198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:32.180117 14198 log.cc:824] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:32.180907 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: No bootstrap required, opened a new log
I20260926 04:05:32.182989 14198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.183140 14198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:32.183202 14198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e6de8508a343c39c734fd4259c9fe3, State: Initialized, Role: FOLLOWER
I20260926 04:05:32.183362 14198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.183661 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.183723 14198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:32.183799 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 04:05:32.185923 14400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:05:32.186038 14400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:05:32.188499 14130 master_runner.cc:263] No leader master found from master 97e6de8508a343c39c734fd4259c9fe3
I20260926 04:05:32.188560 14130 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 04:05:32.223868 14151 tablet.cc:2369] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:05:32.280275 14389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:32.280416 14389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.280651 14389 leader_election.cc:288] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d3e8f82c6764cde867ba166fb925efe (127.0.200.189:34689), 44213ef9a3034a3691792c39f165bd16 (127.0.200.188:43963)
I20260926 04:05:32.281144 14322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97e6de8508a343c39c734fd4259c9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44213ef9a3034a3691792c39f165bd16" is_pre_election: true
I20260926 04:05:32.281312 14322 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e6de8508a343c39c734fd4259c9fe3 in term 0.
I20260926 04:05:32.281654 14149 leader_election.cc:302] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44213ef9a3034a3691792c39f165bd16, 97e6de8508a343c39c734fd4259c9fe3; no voters:
I20260926 04:05:32.281836 14389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:05:32.281885 14389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:05:32.281908 14389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:32.282886 14247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97e6de8508a343c39c734fd4259c9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3e8f82c6764cde867ba166fb925efe" is_pre_election: true
I20260926 04:05:32.283032 14247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e6de8508a343c39c734fd4259c9fe3 in term 0.
I20260926 04:05:32.283635 14389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.283783 14389 leader_election.cc:288] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [CANDIDATE]: Term 1 election: Requested vote from peers 0d3e8f82c6764cde867ba166fb925efe (127.0.200.189:34689), 44213ef9a3034a3691792c39f165bd16 (127.0.200.188:43963)
I20260926 04:05:32.284057 14322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97e6de8508a343c39c734fd4259c9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44213ef9a3034a3691792c39f165bd16"
I20260926 04:05:32.284158 14322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:32.285120 14322 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e6de8508a343c39c734fd4259c9fe3 in term 1.
I20260926 04:05:32.285549 14247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97e6de8508a343c39c734fd4259c9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3e8f82c6764cde867ba166fb925efe"
I20260926 04:05:32.285636 14247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:32.286585 14247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e6de8508a343c39c734fd4259c9fe3 in term 1.
I20260926 04:05:32.286881 14149 leader_election.cc:302] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44213ef9a3034a3691792c39f165bd16, 97e6de8508a343c39c734fd4259c9fe3; no voters:
I20260926 04:05:32.287051 14389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:32.287178 14389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [term 1 LEADER]: Becoming Leader. State: Replica: 97e6de8508a343c39c734fd4259c9fe3, State: Running, Role: LEADER
I20260926 04:05:32.287258 14389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } }
I20260926 04:05:32.287658 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e6de8508a343c39c734fd4259c9fe3. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.287729 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:32.288525 14404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:32.288758 14404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:32.290356 14322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [term 1 FOLLOWER]: Refusing update from remote peer 97e6de8508a343c39c734fd4259c9fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:32.290823 14402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:32.292342 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e6de8508a343c39c734fd4259c9fe3. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.292451 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:05:32.292936 14247 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [term 1 FOLLOWER]: Refusing update from remote peer 97e6de8508a343c39c734fd4259c9fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:32.293236 14402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:32.295512 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e6de8508a343c39c734fd4259c9fe3. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.295622 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:05:32.297163 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.297259 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:32.297503 14375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.297583 14375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:05:32.297854 14405 mvcc.cc:205] Tried to move back new op lower bound from 7333460100258795520 to 7333460100249288704. Current Snapshot: MvccSnapshot[applied={T|T < 7333460100258795520}]
I20260926 04:05:32.297967 14408 mvcc.cc:205] Tried to move back new op lower bound from 7333460100258795520 to 7333460100249288704. Current Snapshot: MvccSnapshot[applied={T|T < 7333460100258795520 or (T in {7333460100258795520})}]
I20260926 04:05:32.298834 14389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.298921 14389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:32.299136 14404 catalog_manager.cc:1347] Generated new cluster ID: 6fbb32f9dc0940a99390e1381d27f360
I20260926 04:05:32.299181 14404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:05:32.303012 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e6de8508a343c39c734fd4259c9fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e6de8508a343c39c734fd4259c9fe3" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37333 } } peers { permanent_uuid: "0d3e8f82c6764cde867ba166fb925efe" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 34689 } } peers { permanent_uuid: "44213ef9a3034a3691792c39f165bd16" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 43963 } } }
I20260926 04:05:32.303119 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:05:32.329087 14404 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:05:32.329829 14404 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:32.336989 14404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e6de8508a343c39c734fd4259c9fe3: Generated new TSK 0
I20260926 04:05:32.337201 14404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:05:33.113445 14365 catalog_manager.cc:1259] Loaded cluster ID: 6fbb32f9dc0940a99390e1381d27f360
I20260926 04:05:33.113557 14365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: loading cluster ID for follower catalog manager: success
I20260926 04:05:33.114573 14365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: acquiring CA information for follower catalog manager: success
I20260926 04:05:33.114993 14365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44213ef9a3034a3691792c39f165bd16: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:05:33.160281 14386 catalog_manager.cc:1259] Loaded cluster ID: 6fbb32f9dc0940a99390e1381d27f360
I20260926 04:05:33.160523 14386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: loading cluster ID for follower catalog manager: success
I20260926 04:05:33.161334 14386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: acquiring CA information for follower catalog manager: success
I20260926 04:05:33.161643 14386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0d3e8f82c6764cde867ba166fb925efe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:05:33.399338 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14130
I20260926 04:05:33.404654 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14205
I20260926 04:05:33.409591 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14280
2026-09-26T04:05:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2994 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:05:33 dist-test-slave-j8c1 krb5kdc[14419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:05:33 dist-test-slave-j8c1 krb5kdc[14419](info): set up 2 sockets
Sep 26 04:05:33 dist-test-slave-j8c1 krb5kdc[14419](info): commencing operation
krb5kdc: starting...
W20260926 04:05:34.454834 802 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:05:34.582494 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:38527
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:34.668645 14432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:34.668882 14432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:34.668907 14432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:34.670110 14432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:34.670159 14432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:34.670174 14432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:34.670192 14432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:34.670209 14432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:34.671752 14432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:38527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:34.671963 14432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:34.672142 14432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:34.677212 14438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:34.677284 14437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:34.677567 14432 server_base.cc:1034] running on GCE node
W20260926 04:05:34.677287 14440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:34.677904 14432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:34.678208 14432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:34.679399 14432 hybrid_clock.cc:648] HybridClock initialized: now 1790395534679383 us; error 29 us; skew 500 ppm
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:34.682211 14432 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:34.682775 14432 webserver.cc:460] Webserver started at http://127.0.200.190:34883/ using document root <none> and password file <none>
I20260926 04:05:34.683027 14432 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:34.683094 14432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:34.683248 14432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:34.684355 14432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "8754ae45a7784383ad7d1005bdc20749"
format_stamp: "Formatted at 2026-09-26 04:05:34 on dist-test-slave-j8c1"
server_key: "5b57053fba7ba6d070c5217051dd043c"
server_key_iv: "d03b900f416dddcf0e3568d46a8ebdb3"
server_key_version: "encryptionkey@0"
I20260926 04:05:34.684741 14432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "8754ae45a7784383ad7d1005bdc20749"
format_stamp: "Formatted at 2026-09-26 04:05:34 on dist-test-slave-j8c1"
server_key: "5b57053fba7ba6d070c5217051dd043c"
server_key_iv: "d03b900f416dddcf0e3568d46a8ebdb3"
server_key_version: "encryptionkey@0"
I20260926 04:05:34.686228 14432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:05:34.687156 14447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:34.687382 14432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:34.687510 14432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "8754ae45a7784383ad7d1005bdc20749"
format_stamp: "Formatted at 2026-09-26 04:05:34 on dist-test-slave-j8c1"
server_key: "5b57053fba7ba6d070c5217051dd043c"
server_key_iv: "d03b900f416dddcf0e3568d46a8ebdb3"
server_key_version: "encryptionkey@0"
I20260926 04:05:34.687593 14432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:34.724869 14432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:34.726933 14432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:34.727113 14432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:34.731626 14432 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:38527
I20260926 04:05:34.731678 14499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:38527 every 8 connection(s)
I20260926 04:05:34.732213 14432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:34.732933 14500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:34.735456 14500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749: Bootstrap starting.
I20260926 04:05:34.736047 14500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:34.736343 14500 log.cc:824] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:34.737103 14500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749: No bootstrap required, opened a new log
I20260926 04:05:34.738421 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14432
I20260926 04:05:34.738536 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:05:34.738791 802 external_mini_cluster.cc:1428] Setting key 717d2f1590518cfa5aef0b5a7bf72e16
I20260926 04:05:34.738883 14500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } }
I20260926 04:05:34.739063 14500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:34.739114 14500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8754ae45a7784383ad7d1005bdc20749, State: Initialized, Role: FOLLOWER
I20260926 04:05:34.739306 14500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } }
I20260926 04:05:34.739413 14500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:34.739451 14500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:34.739530 14500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:34.740108 14500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } }
I20260926 04:05:34.740235 14500 leader_election.cc:302] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8754ae45a7784383ad7d1005bdc20749; no voters:
I20260926 04:05:34.740420 14500 leader_election.cc:288] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:05:34.740484 14505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:34.740645 14505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [term 1 LEADER]: Becoming Leader. State: Replica: 8754ae45a7784383ad7d1005bdc20749, State: Running, Role: LEADER
I20260926 04:05:34.740684 14500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:34.740741 14505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } }
I20260926 04:05:34.741127 14505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8754ae45a7784383ad7d1005bdc20749. Latest consensus state: current_term: 1 leader_uuid: "8754ae45a7784383ad7d1005bdc20749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } } }
I20260926 04:05:34.741283 14505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:34.741511 14514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:34.742020 14514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:34.742475 14506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8754ae45a7784383ad7d1005bdc20749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8754ae45a7784383ad7d1005bdc20749" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 38527 } } }
I20260926 04:05:34.743947 14506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:34.743592 14514 catalog_manager.cc:1347] Generated new cluster ID: aa8c8f8985c5439cacaba97c8bd8dd80
I20260926 04:05:34.744459 14514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:34.750865 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:34.739326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.0.200.190:38527)
0926 04:05:34.739545 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:34.739555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:34.739608 (+ 53us) server_negotiation.cc:407] Connection header received
0926 04:05:34.740018 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:34.740024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:34.740067 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:34.740178 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:34.741056 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:34.742573 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:34.743722 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:34.744279 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:34.744358 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:34.747238 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:34.747242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:34.747243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:34.747254 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:34.749137 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:34.749565 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:34.749568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:34.749568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:34.749680 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:34.749899 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:34.749900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:34.749900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:34.750210 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:34.750353 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:34.750499 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 04:05:34.750664 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
I20260926 04:05:34.752193 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:05:34.758430 14514 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:05:34.759202 14514 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:34.763903 14514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8754ae45a7784383ad7d1005bdc20749: Generated new TSK 0
I20260926 04:05:34.764104 14514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:05:34.806346 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14432
2026-09-26T04:05:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1395 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14527](info): set up 2 sockets
Sep 26 04:05:34 dist-test-slave-j8c1 krb5kdc[14527](info): commencing operation
krb5kdc: starting...
W20260926 04:05:35.846714 802 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:05:35 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:05:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:05:35.970165 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:40447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:36.051407 14540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:36.051621 14540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:36.051644 14540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:36.052748 14540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:36.052786 14540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:36.052803 14540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:36.052816 14540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:36.052829 14540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:36.054242 14540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:40447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:36.054440 14540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:36.054713 14540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:36.059679 14546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.059690 14548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:36.059892 14540 server_base.cc:1034] running on GCE node
W20260926 04:05:36.059701 14545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:36.060227 14540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:36.060523 14540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:36.061668 14540 hybrid_clock.cc:648] HybridClock initialized: now 1790395536061653 us; error 46 us; skew 500 ppm
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:36.073674 14540 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:36.074194 14540 webserver.cc:460] Webserver started at http://127.0.200.190:36559/ using document root <none> and password file <none>
I20260926 04:05:36.074600 14540 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:36.074673 14540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:36.074842 14540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:36.076009 14540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "43b86c5bbb794f51a4a7700ac034c787"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "9c84bcfaeddeb8c9bc47d720d8fadc78"
server_key_iv: "fa121bf8da719cb7ac1e8fefe82ce1e3"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.076314 14540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "43b86c5bbb794f51a4a7700ac034c787"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "9c84bcfaeddeb8c9bc47d720d8fadc78"
server_key_iv: "fa121bf8da719cb7ac1e8fefe82ce1e3"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.077704 14540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:05:36.078474 14555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.078716 14540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:36.078802 14540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "43b86c5bbb794f51a4a7700ac034c787"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "9c84bcfaeddeb8c9bc47d720d8fadc78"
server_key_iv: "fa121bf8da719cb7ac1e8fefe82ce1e3"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.078868 14540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:36.112813 14540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:36.114954 14540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:36.115075 14540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:36.118945 14540 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:40447
I20260926 04:05:36.119026 14607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:40447 every 8 connection(s)
I20260926 04:05:36.119370 14540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:36.120801 14608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:36.123200 14608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap starting.
I20260926 04:05:36.123735 14608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:36.124030 14608 log.cc:824] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:36.124775 14608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: No bootstrap required, opened a new log
I20260926 04:05:36.125774 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14540
I20260926 04:05:36.125864 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:05:36.126080 802 external_mini_cluster.cc:1428] Setting key b6ae96d0c7f492e3966dfd0af2d0f652
I20260926 04:05:36.126868 14608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:36.127056 14608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:36.127092 14608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Initialized, Role: FOLLOWER
I20260926 04:05:36.127226 14608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:36.127324 14608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:36.127358 14608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:36.127415 14608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:36.128187 14608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:36.128307 14608 leader_election.cc:302] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b86c5bbb794f51a4a7700ac034c787; no voters:
I20260926 04:05:36.128473 14608 leader_election.cc:288] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:05:36.128541 14613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:36.128680 14613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 LEADER]: Becoming Leader. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Running, Role: LEADER
I20260926 04:05:36.128719 14608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:36.128764 14613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:36.129073 14613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b86c5bbb794f51a4a7700ac034c787. Latest consensus state: current_term: 1 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:36.129161 14613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:36.129088 14614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:36.129323 14614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:36.129391 14619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:36.129894 14619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:36.131520 14619 catalog_manager.cc:1347] Generated new cluster ID: b25a32d537bf44c494bea40f9d623fd3
I20260926 04:05:36.131588 14619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:36.137358 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.126624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.0.200.190:40447)
0926 04:05:36.126843 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.126853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.126887 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:05:36.126998 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.127004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.127060 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.127204 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:36.128399 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.129903 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.131118 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.131561 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.131725 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.134087 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:36.134092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:36.134092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:36.134104 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:36.135843 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.136245 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.136247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.136248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.136351 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.136568 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.136569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.136569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.136809 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:36.136987 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.137053 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.137189 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":117,"threads_started":1}
I20260926 04:05:36.146811 14619 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:05:36.147521 14619 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:36.153364 14619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Generated new TSK 0
I20260926 04:05:36.153563 14619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:05:36.181562 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:36.284381 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:36.284653 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:36.284698 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:36.285881 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:36.285924 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:36.285951 14634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:36.287649 14634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:36.287935 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:36.288193 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:36.294064 14642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.294139 14639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.294064 14640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:36.294147 14634 server_base.cc:1034] running on GCE node
I20260926 04:05:36.294497 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:36.294778 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:36.295948 14634 hybrid_clock.cc:648] HybridClock initialized: now 1790395536295944 us; error 38 us; skew 500 ppm
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:36.298472 14634 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:36.299000 14634 webserver.cc:460] Webserver started at http://127.0.200.129:35975/ using document root <none> and password file <none>
I20260926 04:05:36.299235 14634 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:36.299304 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:36.299463 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:36.300516 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "9c976e0eb7654e90af9416b904d1bed1"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "6a9de0972a80b782b4acdfa9bedbaf6a"
server_key_iv: "ebe32fb76f82762cafb4a95f5dc6b1c6"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.300876 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "9c976e0eb7654e90af9416b904d1bed1"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "6a9de0972a80b782b4acdfa9bedbaf6a"
server_key_iv: "ebe32fb76f82762cafb4a95f5dc6b1c6"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.302242 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:05:36.303238 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.303438 14634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:36.303546 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "9c976e0eb7654e90af9416b904d1bed1"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "6a9de0972a80b782b4acdfa9bedbaf6a"
server_key_iv: "ebe32fb76f82762cafb4a95f5dc6b1c6"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.303635 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:36.315060 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:36.316999 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:36.317178 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:36.317416 14634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:36.317736 14634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:36.317792 14634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.317842 14634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:36.317873 14634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.323995 14634 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:45553
I20260926 04:05:36.324038 14762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:45553 every 8 connection(s)
I20260926 04:05:36.324404 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:05:36.327380 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14634
I20260926 04:05:36.327493 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:05:36.327677 802 external_mini_cluster.cc:1428] Setting key 40b7cabd00aa9da89e86f58394f18540
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:36.335319 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.325217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:34851 (local address 127.0.200.190:40447)
0926 04:05:36.325264 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.325266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.325514 (+ 248us) server_negotiation.cc:407] Connection header received
0926 04:05:36.326001 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.326002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.326018 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.326090 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:36.327236 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.327972 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.329642 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.329846 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.329900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.332330 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:36.332332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:36.332332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:36.332341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:36.333773 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.334229 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.334230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.334230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.334292 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.334620 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.334621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.334622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.334747 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:36.334828 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.335121 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.335214 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:05:36.335497 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.325163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:34851)
0926 04:05:36.325336 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.325382 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.325908 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.326168 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.326172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.326779 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:36.327017 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.327022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.328149 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.328151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.329557 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.329560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.329938 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.329939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.330102 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.330749 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:36.330757 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:36.332215 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:36.333883 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.333891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.333894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.334131 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.334389 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.334390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.334391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.334510 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.334825 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:36.334833 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:36.334978 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.335086 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.335251 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:05:36.336021 14763 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:36.336119 14763 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:36.336329 14763 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:36.336840 14572 ts_manager.cc:194] Registered new tserver with Master: 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553)
I20260926 04:05:36.337479 14572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:34851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:05:36.353966 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:36.433530 14768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:36.433735 14768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:36.433758 14768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:36.434841 14768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:36.434876 14768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:36.434902 14768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:36.436250 14768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:36.436419 14768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:36.436597 14768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:36.441535 14776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.441535 14773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.441596 14774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:36.441807 14768 server_base.cc:1034] running on GCE node
I20260926 04:05:36.442019 14768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:36.442253 14768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:36.443418 14768 hybrid_clock.cc:648] HybridClock initialized: now 1790395536443403 us; error 28 us; skew 500 ppm
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:36.445895 14768 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:36.446341 14768 webserver.cc:460] Webserver started at http://127.0.200.130:34541/ using document root <none> and password file <none>
I20260926 04:05:36.446633 14768 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:36.446703 14768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:36.446861 14768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:36.447909 14768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "28bfda48fc5e4745a76f92a913a243c0"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "4deebfcd966a6a953385c068094e8460"
server_key_iv: "0c1d80e2242c21fa3c36179a3a723d34"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.448283 14768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "28bfda48fc5e4745a76f92a913a243c0"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "4deebfcd966a6a953385c068094e8460"
server_key_iv: "0c1d80e2242c21fa3c36179a3a723d34"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.449627 14768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:05:36.450356 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.450536 14768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:05:36.450666 14768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "28bfda48fc5e4745a76f92a913a243c0"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "4deebfcd966a6a953385c068094e8460"
server_key_iv: "0c1d80e2242c21fa3c36179a3a723d34"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.450757 14768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:36.469976 14768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:36.471973 14768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:36.472148 14768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:36.472384 14768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:36.472699 14768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:36.472755 14768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.472797 14768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:36.472826 14768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.478693 14768 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:44377
I20260926 04:05:36.478771 14896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:44377 every 8 connection(s)
I20260926 04:05:36.479099 14768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:36.488826 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14768
I20260926 04:05:36.488937 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:05:36.489041 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.480067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44601 (local address 127.0.200.190:40447)
0926 04:05:36.480126 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.480128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.480235 (+ 107us) server_negotiation.cc:407] Connection header received
0926 04:05:36.481008 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.481010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.481022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.481079 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:36.482187 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.482772 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.483796 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.484002 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.484090 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.486280 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:36.486282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:36.486282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:36.486290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:36.487622 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.488057 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.488059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.488059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.488119 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.488370 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.488371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.488371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.488486 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:36.488566 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.488855 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.488949 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:05:36.489149 802 external_mini_cluster.cc:1428] Setting key 67c495e7bc4040bf19afea422364ae4a
I20260926 04:05:36.489238 14899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.479818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:44601)
0926 04:05:36.480005 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.480056 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.480867 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.481214 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.481217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.481871 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:36.482079 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.482083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.482886 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.482887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.483689 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.483691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.484097 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.484098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.484224 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.484746 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:36.484755 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:36.486184 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:36.487721 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.487728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.487730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.487964 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.488197 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.488198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.488198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.488304 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.488556 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:36.488563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:36.488720 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.488824 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.488995 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260926 04:05:36.489878 14897 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:36.489972 14897 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:36.490161 14897 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:36.490603 14572 ts_manager.cc:194] Registered new tserver with Master: 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
I20260926 04:05:36.490974 14572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:44601
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:05:36.513794 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:36.596288 14902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:36.596521 14902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:36.596549 14902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:36.597756 14902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:36.597807 14902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:36.597836 14902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:36.599407 14902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:36.599677 14902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:36.599932 14902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:36.604974 14908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.605018 14910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:36.604974 14907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:36.605342 14902 server_base.cc:1034] running on GCE node
I20260926 04:05:36.605602 14902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:36.605859 14902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:36.607030 14902 hybrid_clock.cc:648] HybridClock initialized: now 1790395536607007 us; error 32 us; skew 500 ppm
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:36.609545 14902 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:36.610082 14902 webserver.cc:460] Webserver started at http://127.0.200.131:34811/ using document root <none> and password file <none>
I20260926 04:05:36.610320 14902 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:36.610390 14902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:36.610600 14902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:36.611685 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "990554ad803a4e10b0a1b3a48450bbe5"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "5bace68786c9ad19de41d21bd67509ff"
server_key_iv: "20e87dac2c0dd7bf28b8293c7cdc2278"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.612068 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "990554ad803a4e10b0a1b3a48450bbe5"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "5bace68786c9ad19de41d21bd67509ff"
server_key_iv: "20e87dac2c0dd7bf28b8293c7cdc2278"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.613446 14902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:05:36.614226 14917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.614420 14902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:36.614543 14902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "990554ad803a4e10b0a1b3a48450bbe5"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "5bace68786c9ad19de41d21bd67509ff"
server_key_iv: "20e87dac2c0dd7bf28b8293c7cdc2278"
server_key_version: "encryptionkey@0"
I20260926 04:05:36.614663 14902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:36.632020 14902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:36.634114 14902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:36.634284 14902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:36.634531 14902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:36.634914 14902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:36.634970 14902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.635020 14902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:36.635048 14902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:36.641273 14902 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:33959
I20260926 04:05:36.641315 15030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:33959 every 8 connection(s)
I20260926 04:05:36.641645 14902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
Sep 26 04:05:36 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:36.649041 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 14902
I20260926 04:05:36.649132 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:05:36.649326 802 external_mini_cluster.cc:1428] Setting key 7186ccadace38733f46bf831fc5f23d5
I20260926 04:05:36.651345 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.642411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:58249 (local address 127.0.200.190:40447)
0926 04:05:36.642476 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.642478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.642740 (+ 262us) server_negotiation.cc:407] Connection header received
0926 04:05:36.643284 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.643285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.643296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.643384 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:36.644380 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.644885 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.645845 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.646047 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.646104 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.648396 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:36.648398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:36.648398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:36.648405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:36.649969 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.650451 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.650452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.650453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.650514 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.650800 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.650800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.650801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.650908 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:36.650962 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.651216 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.651280 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 04:05:36.651585 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.642391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:58249)
0926 04:05:36.642598 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.642643 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.643180 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.643465 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.643468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.644083 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:36.644279 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.644284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.644969 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.644969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.645766 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.645768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.646126 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.646127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.646229 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.646820 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:36.646828 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:36.648305 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:36.650068 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.650077 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.650079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.650348 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.650618 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:36.650619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:36.650619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:36.650738 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:36.650979 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:36.650984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:36.651101 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.651187 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.651309 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260926 04:05:36.652021 15031 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:36.652107 15031 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:36.652287 15031 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:36.652664 14572 ts_manager.cc:194] Registered new tserver with Master: 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959)
I20260926 04:05:36.653040 14572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:58249
I20260926 04:05:36.661355 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:05:36.668824 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.662703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.0.200.190:40447)
0926 04:05:36.662743 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.662746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.662880 (+ 134us) server_negotiation.cc:407] Connection header received
0926 04:05:36.662958 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.662959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.662969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.663014 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:36.663580 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.664063 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.664788 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.664986 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.665074 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.666303 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:36.666305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:36.666305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:36.666313 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:36.667579 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.667939 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.667940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.667941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.668003 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:36.668220 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:36.668221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:36.668221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:36.668378 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:36.668483 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.668627 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.668716 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 04:05:36.670444 14572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:05:36.672142 14572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:05:36.678813 15046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:33959 (local address 127.0.0.1:44144)
0926 04:05:36.675147 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.675157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.675195 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.675475 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.675476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.675478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:36.675519 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.675520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676819 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.676820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.677775 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.677778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678469 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.678581 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678599 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.678667 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.678726 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":63,"threads_started":1}
I20260926 04:05:36.678938 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.0.200.130:44377)
0926 04:05:36.675075 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.675080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.675097 (+ 17us) server_negotiation.cc:407] Connection header received
0926 04:05:36.675144 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.675145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.675158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.675245 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:36.675649 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676318 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.677728 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678497 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678591 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678651 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.678778 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.678827 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":146,"threads_started":1}
I20260926 04:05:36.679086 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.0.200.131:33959)
0926 04:05:36.675308 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.675313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.675323 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:05:36.675357 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.675360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.675375 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.675466 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:36.675581 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676692 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.677867 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678369 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678457 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678518 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.678944 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.679011 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":94,"threads_started":1}
I20260926 04:05:36.679169 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45553 (local address 127.0.0.1:49294)
0926 04:05:36.674915 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.674959 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.675029 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.675388 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.675389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.675391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:36.675444 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.675446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676791 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.676792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.677813 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.677815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678807 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.678866 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678870 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.679023 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.679074 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":3,"thread_start_us":49,"threads_started":1}
I20260926 04:05:36.679323 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:44377 (local address 127.0.0.1:34014)
0926 04:05:36.674919 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.674960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.675041 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.675249 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.675253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.675255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:36.675324 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.675326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676417 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.676418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.677637 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.677640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678621 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.678687 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678690 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.678741 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.678773 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":82,"threads_started":1}
I20260926 04:05:36.679427 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.674763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.0.200.129:45553)
0926 04:05:36.675146 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.675151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.675163 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:05:36.675206 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.675209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.675225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.675298 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:36.675524 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.676675 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.677896 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.678634 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.678693 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.678771 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.679295 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.679341 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":77,"threads_started":1}
I20260926 04:05:36.679867 14965 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6e8488fd174a159e67b06c336b856d (DEFAULT_TABLE table=test-table [id=c7f097d42ef045528705db508729c104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:36.680132 14831 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6e8488fd174a159e67b06c336b856d (DEFAULT_TABLE table=test-table [id=c7f097d42ef045528705db508729c104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:36.680275 14965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6e8488fd174a159e67b06c336b856d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:36.680243 14697 tablet_service.cc:1467] Processing CreateTablet for tablet 9e6e8488fd174a159e67b06c336b856d (DEFAULT_TABLE table=test-table [id=c7f097d42ef045528705db508729c104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:36.680426 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6e8488fd174a159e67b06c336b856d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:36.680609 14697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e6e8488fd174a159e67b06c336b856d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:36.683568 15048 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap starting.
I20260926 04:05:36.684012 15050 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Bootstrap starting.
I20260926 04:05:36.684257 15048 tablet_bootstrap.cc:654] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:36.684619 15050 tablet_bootstrap.cc:654] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:36.684813 15048 log.cc:824] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:36.685190 15050 log.cc:824] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:36.685556 15049 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Bootstrap starting.
I20260926 04:05:36.685808 15048 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: No bootstrap required, opened a new log
I20260926 04:05:36.685887 15048 ts_tablet_manager.cc:1397] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:36.686012 15050 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: No bootstrap required, opened a new log
I20260926 04:05:36.686076 15050 ts_tablet_manager.cc:1397] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:05:36.686276 15049 tablet_bootstrap.cc:654] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:36.686838 15049 log.cc:824] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:36.687644 15049 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: No bootstrap required, opened a new log
I20260926 04:05:36.687732 15049 ts_tablet_manager.cc:1397] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:36.688143 15050 raft_consensus.cc:348] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.688143 15048 raft_consensus.cc:348] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.688333 15048 raft_consensus.cc:374] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:36.688331 15050 raft_consensus.cc:374] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:36.688369 15048 raft_consensus.cc:729] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c976e0eb7654e90af9416b904d1bed1, State: Initialized, Role: FOLLOWER
I20260926 04:05:36.688369 15050 raft_consensus.cc:729] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990554ad803a4e10b0a1b3a48450bbe5, State: Initialized, Role: FOLLOWER
I20260926 04:05:36.688498 15050 consensus_queue.cc:260] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.688544 15048 consensus_queue.cc:260] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.688783 15050 ts_tablet_manager.cc:1428] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:05:36.688786 15048 ts_tablet_manager.cc:1428] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:05:36.688846 14763 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
I20260926 04:05:36.688953 15031 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
I20260926 04:05:36.689716 15049 raft_consensus.cc:348] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.689908 15049 raft_consensus.cc:374] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:36.689958 15049 raft_consensus.cc:729] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bfda48fc5e4745a76f92a913a243c0, State: Initialized, Role: FOLLOWER
I20260926 04:05:36.690109 15049 consensus_queue.cc:260] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.690377 15049 ts_tablet_manager.cc:1428] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:05:36.690426 14897 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
W20260926 04:05:36.729666 14898 tablet.cc:2369] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:05:36.825192 14764 tablet.cc:2369] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:05:36.892932 15032 tablet.cc:2369] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:05:36.971710 15055 raft_consensus.cc:482] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:36.971840 15055 raft_consensus.cc:504] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.972258 15055 leader_election.cc:288] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377), 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959)
I20260926 04:05:36.976351 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.972549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:40377 (local address 127.0.200.130:44377)
0926 04:05:36.972605 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.972608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.972731 (+ 123us) server_negotiation.cc:407] Connection header received
0926 04:05:36.972788 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.972789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.972799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.972849 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:36.973085 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.973845 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.975365 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.975937 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.976016 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.976025 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.976239 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.976279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:05:36.976351 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.972614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:37149 (local address 127.0.200.131:33959)
0926 04:05:36.972672 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:36.972674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:36.972692 (+ 18us) server_negotiation.cc:407] Connection header received
0926 04:05:36.972771 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:36.972772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:36.972783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:36.972849 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:36.973086 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.973845 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.975368 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.975937 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.976017 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.976025 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:36.976239 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 04:05:36.976279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 04:05:36.976382 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.972472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:33959 (local address 127.0.200.129:37149)
0926 04:05:36.972620 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.972631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.972707 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.972882 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.972883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.972886 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:36.973001 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.973004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.973945 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.973946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.975258 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.975261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.976059 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.976059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.976145 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.976151 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.976215 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.976273 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":17,"thread_start_us":61,"threads_started":1}
I20260926 04:05:36.976384 15057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:36.972485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:44377 (local address 127.0.200.129:40377)
0926 04:05:36.972657 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:36.972665 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:36.972717 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:36.972883 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:36.972884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:36.972887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:36.973006 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.973007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.973944 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.973946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.975258 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:36.975261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:36.976057 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:36.976058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:36.976146 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:36.976151 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:05:36.976215 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 04:05:36.976273 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":99,"thread_start_us":54,"threads_started":1}
I20260926 04:05:36.976796 14985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6e8488fd174a159e67b06c336b856d" candidate_uuid: "9c976e0eb7654e90af9416b904d1bed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990554ad803a4e10b0a1b3a48450bbe5" is_pre_election: true
I20260926 04:05:36.976962 14985 raft_consensus.cc:2454] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c976e0eb7654e90af9416b904d1bed1 in term 0.
I20260926 04:05:36.977283 14652 leader_election.cc:302] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 990554ad803a4e10b0a1b3a48450bbe5, 9c976e0eb7654e90af9416b904d1bed1; no voters:
I20260926 04:05:36.977263 14851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6e8488fd174a159e67b06c336b856d" candidate_uuid: "9c976e0eb7654e90af9416b904d1bed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bfda48fc5e4745a76f92a913a243c0" is_pre_election: true
I20260926 04:05:36.977427 14851 raft_consensus.cc:2454] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c976e0eb7654e90af9416b904d1bed1 in term 0.
I20260926 04:05:36.977425 15055 raft_consensus.cc:2781] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:05:36.977514 15055 raft_consensus.cc:482] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:05:36.977552 15055 raft_consensus.cc:3037] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:36.978269 15055 raft_consensus.cc:504] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.978425 15055 leader_election.cc:288] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 1 election: Requested vote from peers 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377), 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959)
I20260926 04:05:36.978643 14985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6e8488fd174a159e67b06c336b856d" candidate_uuid: "9c976e0eb7654e90af9416b904d1bed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990554ad803a4e10b0a1b3a48450bbe5"
I20260926 04:05:36.978657 14851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e6e8488fd174a159e67b06c336b856d" candidate_uuid: "9c976e0eb7654e90af9416b904d1bed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bfda48fc5e4745a76f92a913a243c0"
I20260926 04:05:36.978718 14851 raft_consensus.cc:3037] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:36.978729 14985 raft_consensus.cc:3037] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:36.979400 14985 raft_consensus.cc:2454] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c976e0eb7654e90af9416b904d1bed1 in term 1.
I20260926 04:05:36.979449 14851 raft_consensus.cc:2454] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c976e0eb7654e90af9416b904d1bed1 in term 1.
I20260926 04:05:36.979616 14652 leader_election.cc:302] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 990554ad803a4e10b0a1b3a48450bbe5, 9c976e0eb7654e90af9416b904d1bed1; no voters:
I20260926 04:05:36.979738 15055 raft_consensus.cc:2781] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:36.979889 15055 raft_consensus.cc:686] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 LEADER]: Becoming Leader. State: Replica: 9c976e0eb7654e90af9416b904d1bed1, State: Running, Role: LEADER
I20260926 04:05:36.980029 15055 consensus_queue.cc:237] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:36.980885 14572 catalog_manager.cc:5495] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129). New cstate: current_term: 1 leader_uuid: "9c976e0eb7654e90af9416b904d1bed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } health_report { overall_health: UNKNOWN } } }
I20260926 04:05:37.035483 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.032994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.0.200.129:45553)
0926 04:05:37.033066 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.033069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.033077 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:37.033123 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.033124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.033142 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.033221 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:05:37.033390 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.033884 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.034700 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.034895 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.034973 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.035131 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:05:37.035246 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:05:37.035344 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.035363 (+ 19us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.035385 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:05:37.037403 14985 raft_consensus.cc:1261] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Refusing update from remote peer 9c976e0eb7654e90af9416b904d1bed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:37.037436 14851 raft_consensus.cc:1261] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Refusing update from remote peer 9c976e0eb7654e90af9416b904d1bed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:37.037799 15055 consensus_queue.cc:1035] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:37.037961 15059 consensus_queue.cc:1035] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:37.040450 15063 mvcc.cc:205] Tried to move back new op lower bound from 7333460119702294528 to 7333460119470755840. Current Snapshot: MvccSnapshot[applied={T|T < 7333460119702294528}]
I20260926 04:05:37.040210 15062 mvcc.cc:205] Tried to move back new op lower bound from 7333460119702294528 to 7333460119470755840. Current Snapshot: MvccSnapshot[applied={T|T < 7333460119702294528}]
I20260926 04:05:37.043388 14572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:37.043488 14572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:37.044258 14572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Sending DeleteTablet for 3 replicas of tablet 9e6e8488fd174a159e67b06c336b856d
I20260926 04:05:37.044626 14697 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e6e8488fd174a159e67b06c336b856d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:49294
I20260926 04:05:37.044626 14831 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e6e8488fd174a159e67b06c336b856d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:34014
I20260926 04:05:37.044693 14965 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e6e8488fd174a159e67b06c336b856d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:44144
I20260926 04:05:37.044881 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14634
I20260926 04:05:37.044917 15072 tablet_replica.cc:330] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: stopping tablet replica
I20260926 04:05:37.045014 15072 raft_consensus.cc:2229] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:05:37.045150 15072 raft_consensus.cc:2258] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 04:05:37.050693 14557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:45553 (error 108)
I20260926 04:05:37.050720 15073 tablet_replica.cc:330] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: stopping tablet replica
I20260926 04:05:37.050812 15074 tablet_replica.cc:330] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: stopping tablet replica
I20260926 04:05:37.050845 15073 raft_consensus.cc:2229] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.050962 15073 raft_consensus.cc:2258] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.051033 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14768
I20260926 04:05:37.051043 15074 raft_consensus.cc:2229] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.051620 15074 raft_consensus.cc:2258] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.052098 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.051833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45553 (local address 127.0.0.1:49318)
0926 04:05:37.052009 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.052052 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260926 04:05:37.052213 15074 ts_tablet_manager.cc:1905] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 04:05:37.052814 14557 catalog_manager.cc:4584] TS 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e6e8488fd174a159e67b06c336b856d: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45553: connect: Connection refused (error 111)
I20260926 04:05:37.054418 15074 ts_tablet_manager.cc:1918] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:37.054517 15074 log.cc:1196] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/9e6e8488fd174a159e67b06c336b856d
I20260926 04:05:37.054812 15074 ts_tablet_manager.cc:1939] T 9e6e8488fd174a159e67b06c336b856d P 990554ad803a4e10b0a1b3a48450bbe5: Deleting consensus metadata
I20260926 04:05:37.055249 14558 catalog_manager.cc:4841] TS 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): tablet 9e6e8488fd174a159e67b06c336b856d (table test-table [id=c7f097d42ef045528705db508729c104]) successfully deleted
I20260926 04:05:37.056211 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14902
I20260926 04:05:37.056269 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.056149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:44377 (local address 127.0.0.1:34020)
0926 04:05:37.056202 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.056235 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.130:44377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 04:05:37.060864 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 14540
I20260926 04:05:37.066636 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--webserver_interface=127.0.200.190
--webserver_port=36559
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:40447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:37.147792 15076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.147995 15076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.148017 15076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.149091 15076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:37.149121 15076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.149137 15076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.149149 15076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:37.149161 15076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:37.150539 15076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:40447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=36559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.150781 15076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.151000 15076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.159606 15083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.159782 15076 server_base.cc:1034] running on GCE node
W20260926 04:05:37.159832 15085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.159648 15082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.160264 15076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.160619 15076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.161772 15076 hybrid_clock.cc:648] HybridClock initialized: now 1790395537161763 us; error 28 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.164472 15076 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:37.164932 15076 webserver.cc:460] Webserver started at http://127.0.200.190:36559/ using document root <none> and password file <none>
I20260926 04:05:37.165179 15076 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.165274 15076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.166771 15076 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.167407 15092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.167580 15076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.167721 15076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "43b86c5bbb794f51a4a7700ac034c787"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "9c84bcfaeddeb8c9bc47d720d8fadc78"
server_key_iv: "fa121bf8da719cb7ac1e8fefe82ce1e3"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.168128 15076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.176101 15076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.178397 15076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.178654 15076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.182674 15076 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:40447
I20260926 04:05:37.182708 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:40447 every 8 connection(s)
I20260926 04:05:37.183136 15076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:37.184737 15145 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:05:37.185403 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap starting.
I20260926 04:05:37.191507 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15076
I20260926 04:05:37.191802 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:45553
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=35975
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:37.193885 15145 log.cc:824] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.196841 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:37.197242 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap complete.
I20260926 04:05:37.201076 15145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.201476 15145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.201684 15145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.201767 15145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:37.201838 15145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:37.201893 15145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:05:37.203475 15145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.203595 15145 leader_election.cc:302] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b86c5bbb794f51a4a7700ac034c787; no voters:
I20260926 04:05:37.203802 15145 leader_election.cc:288] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 04:05:37.203862 15149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 04:05:37.203990 15149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 LEADER]: Becoming Leader. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Running, Role: LEADER
I20260926 04:05:37.204088 15149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.204285 15145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:37.204339 15149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b86c5bbb794f51a4a7700ac034c787. Latest consensus state: current_term: 2 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:37.204342 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:37.204416 15149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:37.204492 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:37.204933 15153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:37.205892 15153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:37.206056 15153 catalog_manager.cc:1259] Loaded cluster ID: b25a32d537bf44c494bea40f9d623fd3
I20260926 04:05:37.206353 15153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:05:37.206668 15153 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 04:05:37.209334 15153 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 04:05:37.211251 15153 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:37.211452 15153 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Loaded TSK: 0
I20260926 04:05:37.211680 15153 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 04:05:37.289499 15147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.289754 15147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.289798 15147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.291167 15147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.291256 15147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.291320 15147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:37.293690 15147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:45553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=35975
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.293991 15147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.294274 15147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.299487 15172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.299489 15173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.299489 15175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.299973 15147 server_base.cc:1034] running on GCE node
I20260926 04:05:37.300240 15147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.300503 15147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.301667 15147 hybrid_clock.cc:648] HybridClock initialized: now 1790395537301650 us; error 33 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.304383 15147 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:37.304888 15147 webserver.cc:460] Webserver started at http://127.0.200.129:35975/ using document root <none> and password file <none>
I20260926 04:05:37.305205 15147 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.305289 15147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.306730 15147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.307444 15182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.307619 15147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.307716 15147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "9c976e0eb7654e90af9416b904d1bed1"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "6a9de0972a80b782b4acdfa9bedbaf6a"
server_key_iv: "ebe32fb76f82762cafb4a95f5dc6b1c6"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.308072 15147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.344179 15147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.346230 15147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.346411 15147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.346695 15147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:37.347128 15189 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 04:05:37.348692 15147 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 04:05:37.348757 15147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 04:05:37.348807 15147 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 04:05:37.349488 15147 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 04:05:37.349545 15147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:37.349628 15189 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap starting.
I20260926 04:05:37.356357 15147 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:45553
I20260926 04:05:37.356454 15296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:45553 every 8 connection(s)
I20260926 04:05:37.356752 15147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:05:37.358642 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15147
I20260926 04:05:37.358937 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:44377
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=34541
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:37.360351 15189 log.cc:824] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.362877 15189 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:37.363181 15189 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap complete.
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:37.365284 15189 ts_tablet_manager.cc:1397] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260926 04:05:37.366993 15189 raft_consensus.cc:348] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:37.367393 15189 raft_consensus.cc:729] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c976e0eb7654e90af9416b904d1bed1, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.367569 15189 consensus_queue.cc:260] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:37.367875 15189 ts_tablet_manager.cc:1428] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:05:37.369303 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.357670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:52847 (local address 127.0.200.190:40447)
0926 04:05:37.357886 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.357897 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.357954 (+ 57us) server_negotiation.cc:407] Connection header received
0926 04:05:37.358619 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.358631 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.358699 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.358863 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.359751 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.361042 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.362354 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.362827 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.363020 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.365650 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.365655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.365655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.365679 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.367448 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.367985 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.367988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.367988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.368084 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.368345 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.368346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.368347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.368579 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.368665 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.369001 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.369136 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260926 04:05:37.369302 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.357522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:52847)
0926 04:05:37.357733 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.357789 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.358342 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.358901 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.358906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.359324 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:37.359602 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.359609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.361171 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.361172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.362255 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.362258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.363035 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.363037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.363189 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.363817 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:37.363825 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:37.365544 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:37.367540 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.367546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.367549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.367849 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.368161 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.368162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.368162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.368269 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.368660 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:37.368667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:37.368804 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.368907 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.369101 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260926 04:05:37.370303 15297 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:37.370397 15297 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:37.370610 15297 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:37.371330 15109 ts_manager.cc:194] Registered new tserver with Master: 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553)
I20260926 04:05:37.372313 15109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:52847
I20260926 04:05:37.373128 15297 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.129@KRBTEST.COM
I20260926 04:05:37.380298 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.371968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.0.200.129:45553)
0926 04:05:37.372309 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.372314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.372323 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:37.372352 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.372354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.372371 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.372428 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.373013 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.373922 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.375026 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.375302 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.375362 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.377535 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.377548 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.377548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.377554 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.379042 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.379446 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.379449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.379450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.379507 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.379704 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.379705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.379706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.379840 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.379923 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.380104 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.380200 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":86,"threads_started":1}
I20260926 04:05:37.380298 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.371911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45553 (local address 127.0.0.1:49320)
0926 04:05:37.372107 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.372137 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.372194 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.372485 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.372488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.372875 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:37.372929 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.372931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.373993 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.373994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.374940 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.374943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.375377 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.375378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.375454 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.375928 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:37.375933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:37.377440 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:37.379146 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.379152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.379154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.379351 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.379573 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.379574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.379574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.379633 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.379915 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:37.379920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:37.379971 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.380071 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.380201 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
I20260926 04:05:37.380687 15231 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e6e8488fd174a159e67b06c336b856d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260926 04:05:37.381224 15306 tablet_replica.cc:330] stopping tablet replica
I20260926 04:05:37.381333 15306 raft_consensus.cc:2229] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.381398 15306 raft_consensus.cc:2258] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.381708 15306 ts_tablet_manager.cc:1905] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:37.384034 15306 ts_tablet_manager.cc:1918] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:37.384106 15306 log.cc:1196] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/9e6e8488fd174a159e67b06c336b856d
I20260926 04:05:37.384328 15306 ts_tablet_manager.cc:1939] T 9e6e8488fd174a159e67b06c336b856d P 9c976e0eb7654e90af9416b904d1bed1: Deleting consensus metadata
I20260926 04:05:37.384824 15094 catalog_manager.cc:4841] TS 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553): tablet 9e6e8488fd174a159e67b06c336b856d (table test-table [id=c7f097d42ef045528705db508729c104]) successfully deleted
W20260926 04:05:37.446367 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.446592 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.446661 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.447767 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.447805 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.447831 15301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:37.449204 15301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=34541
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.449396 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.449572 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.454692 15311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.454681 15312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.454682 15314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.454707 15301 server_base.cc:1034] running on GCE node
I20260926 04:05:37.455320 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.455647 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.456867 15301 hybrid_clock.cc:648] HybridClock initialized: now 1790395537456846 us; error 32 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.459555 15301 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:37.460021 15301 webserver.cc:460] Webserver started at http://127.0.200.130:34541/ using document root <none> and password file <none>
I20260926 04:05:37.460254 15301 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.460340 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.461874 15301 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:05:37.462782 15321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.462986 15301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.463075 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "28bfda48fc5e4745a76f92a913a243c0"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "4deebfcd966a6a953385c068094e8460"
server_key_iv: "0c1d80e2242c21fa3c36179a3a723d34"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.463416 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.474931 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.477259 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.477373 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.477566 15301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:37.477933 15328 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 04:05:37.479465 15301 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 04:05:37.479511 15301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 04:05:37.479535 15301 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 04:05:37.480115 15301 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 04:05:37.480152 15301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:37.480232 15328 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Bootstrap starting.
I20260926 04:05:37.486573 15301 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:44377
I20260926 04:05:37.486685 15435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:44377 every 8 connection(s)
I20260926 04:05:37.487018 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:05:37.490749 15328 log.cc:824] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.493287 15328 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:37.493604 15328 tablet_bootstrap.cc:492] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Bootstrap complete.
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:37.494638 15328 ts_tablet_manager.cc:1397] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.006s
I20260926 04:05:37.495666 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15301
I20260926 04:05:37.495967 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:33959
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=34811
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:37.495812 15328 raft_consensus.cc:348] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:37.496165 15328 raft_consensus.cc:729] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bfda48fc5e4745a76f92a913a243c0, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.496312 15328 consensus_queue.cc:260] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } }
I20260926 04:05:37.496629 15328 ts_tablet_manager.cc:1428] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 04:05:37.498446 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.487974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:58447 (local address 127.0.200.190:40447)
0926 04:05:37.488054 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.488057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.488321 (+ 264us) server_negotiation.cc:407] Connection header received
0926 04:05:37.489122 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.489123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.489138 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.489197 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.490155 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.490754 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.492140 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.492352 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.492414 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.494991 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.494993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.494993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.495002 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.496930 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.497357 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.497359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.497359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.497432 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.497744 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.497745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.497745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.497859 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.497911 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.498279 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.498373 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:05:37.498644 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.487865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:58447)
0926 04:05:37.488143 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.488215 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.488967 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.489277 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.489283 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.489683 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:37.490022 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.490032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.490865 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.490866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.492046 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.492048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.492459 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.492460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.492607 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.493131 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:37.493141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:37.494891 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:37.497021 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.497026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.497028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.497265 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.497510 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.497511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.497511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.497652 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.497926 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:37.497932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:37.498097 (+ 165us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.498215 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.498407 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":111,"threads_started":1}
I20260926 04:05:37.499136 15436 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:37.499269 15436 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:37.499511 15436 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:37.500187 15109 ts_manager.cc:194] Registered new tserver with Master: 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
I20260926 04:05:37.500826 15109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:58447
I20260926 04:05:37.501578 15436 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.130@KRBTEST.COM
I20260926 04:05:37.509300 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.500526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:44377 (local address 127.0.0.1:34036)
0926 04:05:37.500569 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.500580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.500620 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.500860 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.500861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.501088 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:37.501154 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.501157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.502634 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.502635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.503233 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.503236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.503914 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.503915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.503969 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.504536 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:37.504542 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:37.506045 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:37.508295 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.508296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.508296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.508481 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.508699 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.508699 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.508700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.508754 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.509051 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:37.509052 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:37.509088 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.509145 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.509228 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260926 04:05:37.509373 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.500566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.0.200.130:44377)
0926 04:05:37.500721 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.500731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.500740 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:37.500766 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.500767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.500781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.500859 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.501241 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.502489 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.503326 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.503827 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.503911 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.506131 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.506139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.506139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.506148 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.508203 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.508568 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.508571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.508571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.508632 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.508814 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.508815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.508815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.508984 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.509063 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.509183 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.509277 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260926 04:05:37.509902 15370 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e6e8488fd174a159e67b06c336b856d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:34036
I20260926 04:05:37.510617 15443 tablet_replica.cc:330] stopping tablet replica
I20260926 04:05:37.510735 15443 raft_consensus.cc:2229] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.510807 15443 raft_consensus.cc:2258] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.511101 15443 ts_tablet_manager.cc:1905] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:37.513211 15443 ts_tablet_manager.cc:1918] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:37.513307 15443 log.cc:1196] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/9e6e8488fd174a159e67b06c336b856d
I20260926 04:05:37.513530 15443 ts_tablet_manager.cc:1939] T 9e6e8488fd174a159e67b06c336b856d P 28bfda48fc5e4745a76f92a913a243c0: Deleting consensus metadata
I20260926 04:05:37.514048 15096 catalog_manager.cc:4841] TS 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): tablet 9e6e8488fd174a159e67b06c336b856d (table test-table [id=c7f097d42ef045528705db508729c104]) successfully deleted
W20260926 04:05:37.582122 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.582324 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.582347 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.583535 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.583582 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.583609 15441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:37.585069 15441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:33959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=34811
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.585328 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.585558 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.590691 15448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.590683 15449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.590683 15451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.590914 15441 server_base.cc:1034] running on GCE node
I20260926 04:05:37.591142 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.591399 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.592559 15441 hybrid_clock.cc:648] HybridClock initialized: now 1790395537592535 us; error 42 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.594981 15441 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:37.595476 15441 webserver.cc:460] Webserver started at http://127.0.200.131:34811/ using document root <none> and password file <none>
I20260926 04:05:37.595731 15441 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.595813 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.597131 15441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.597712 15458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.597877 15441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.597971 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "990554ad803a4e10b0a1b3a48450bbe5"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "5bace68786c9ad19de41d21bd67509ff"
server_key_iv: "20e87dac2c0dd7bf28b8293c7cdc2278"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.598320 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.608109 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.610172 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.610339 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.610653 15441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:37.611047 15441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:37.611150 15441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.611200 15441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:37.611241 15441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.617146 15441 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:33959
I20260926 04:05:37.617208 15571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:33959 every 8 connection(s)
I20260926 04:05:37.617586 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:05:37.622036 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15441
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:37.627869 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.618368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:34261 (local address 127.0.200.190:40447)
0926 04:05:37.618438 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.618440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.618719 (+ 279us) server_negotiation.cc:407] Connection header received
0926 04:05:37.619297 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.619298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.619312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.619361 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.620347 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.620861 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.621893 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.622093 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.622179 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.624561 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.624562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.624563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.624590 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.626287 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.626816 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.626817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.626818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.626879 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.627145 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.627146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.627146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.627261 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.627337 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.627632 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.627772 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:05:37.628012 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.618372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:34261)
0926 04:05:37.618533 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.618624 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.619173 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.619447 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.619451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.620043 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:37.620243 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.620247 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.620959 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.620960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.621780 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.621782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.622175 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.622176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.622326 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.623038 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:37.623046 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:37.624469 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:37.626397 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.626405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.626407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.626732 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.626946 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:37.626946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:37.626947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:37.627069 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:37.627329 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:37.627336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:37.627480 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.627598 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.627774 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260926 04:05:37.628510 15572 heartbeater.cc:347] Connected to a master server at 127.0.200.190:40447
I20260926 04:05:37.628623 15572 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:37.628804 15572 heartbeater.cc:511] Master 127.0.200.190:40447 requested a full tablet report, sending...
I20260926 04:05:37.629189 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.622476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.0.200.190:40447)
0926 04:05:37.622626 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.622628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.622662 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:05:37.622731 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.622731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.622744 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.622810 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:37.623552 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.624058 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.624969 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.625169 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.625250 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.626347 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:37.626349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:37.626349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:37.626355 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:37.627876 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.628327 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.628328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.628328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.628384 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:37.628567 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:37.628569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:37.628569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:37.628693 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:37.628846 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.628885 (+ 39us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.629079 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260926 04:05:37.629210 15109 ts_manager.cc:194] Registered new tserver with Master: 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959)
I20260926 04:05:37.629645 15109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:34261
I20260926 04:05:37.629943 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:05:37.632524 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.630303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.0.200.190:40447)
0926 04:05:37.630338 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.630339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.630468 (+ 129us) server_negotiation.cc:407] Connection header received
0926 04:05:37.630504 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.630505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.630540 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.630635 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:05:37.630724 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.631152 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.631873 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.632104 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.632174 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.632286 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:05:37.632370 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:05:37.632429 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.632457 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.632476 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 04:05:37.632802 15108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:05:37.634393 15108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:05:37.636917 15370 tablet_service.cc:1467] Processing CreateTablet for tablet 272cef9906a6474c85bb049cfddf89d6 (DEFAULT_TABLE table=test-table [id=133845a0cd244ebabb5c497b0892edb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:37.636917 15231 tablet_service.cc:1467] Processing CreateTablet for tablet 272cef9906a6474c85bb049cfddf89d6 (DEFAULT_TABLE table=test-table [id=133845a0cd244ebabb5c497b0892edb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:37.637126 15370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272cef9906a6474c85bb049cfddf89d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:37.637131 15231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272cef9906a6474c85bb049cfddf89d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:37.638705 15328 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Bootstrap starting.
I20260926 04:05:37.638705 15189 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap starting.
I20260926 04:05:37.639011 15328 tablet_bootstrap.cc:654] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:37.639020 15189 tablet_bootstrap.cc:654] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:37.639695 15328 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: No bootstrap required, opened a new log
I20260926 04:05:37.639747 15328 ts_tablet_manager.cc:1397] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:37.640017 15189 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: No bootstrap required, opened a new log
I20260926 04:05:37.640065 15189 ts_tablet_manager.cc:1397] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.640112 15328 raft_consensus.cc:348] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.640194 15328 raft_consensus.cc:374] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:37.640237 15328 raft_consensus.cc:729] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bfda48fc5e4745a76f92a913a243c0, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.640231 15189 raft_consensus.cc:348] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.640285 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.636612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:33959 (local address 127.0.0.1:44150)
0926 04:05:37.636667 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.636679 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.636717 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.637193 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.637195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.637197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:37.637243 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.637245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.638128 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.638129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.639033 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.639036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.639987 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.639989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.640071 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.640093 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.640159 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.640206 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:05:37.640283 15328 consensus_queue.cc:260] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.640297 15189 raft_consensus.cc:374] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:37.640324 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.636788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.0.200.131:33959)
0926 04:05:37.636968 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.636977 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.636986 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:05:37.637057 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.637060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.637077 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.637319 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:37.637375 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.638031 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.639149 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.639892 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.639972 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.640035 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.640195 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.640239 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260926 04:05:37.640339 15189 raft_consensus.cc:729] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c976e0eb7654e90af9416b904d1bed1, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.640383 15328 ts_tablet_manager.cc:1428] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:37.640383 15189 consensus_queue.cc:260] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.640486 15189 ts_tablet_manager.cc:1428] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.641053 15506 tablet_service.cc:1467] Processing CreateTablet for tablet 272cef9906a6474c85bb049cfddf89d6 (DEFAULT_TABLE table=test-table [id=133845a0cd244ebabb5c497b0892edb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:37.641430 15506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 272cef9906a6474c85bb049cfddf89d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:37.644354 15581 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Bootstrap starting.
I20260926 04:05:37.644575 15440 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:37.644663 15440 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.644999 15440 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553)
I20260926 04:05:37.645093 15581 tablet_bootstrap.cc:654] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:37.645692 15581 log.cc:824] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.646457 15581 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: No bootstrap required, opened a new log
I20260926 04:05:37.646634 15581 ts_tablet_manager.cc:1397] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:37.648440 15581 raft_consensus.cc:348] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.648628 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.645141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:33959 (local address 127.0.200.130:34267)
0926 04:05:37.645211 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.645222 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.645292 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.645869 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.645870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.645871 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:37.645919 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.645920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.646824 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.646826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.647616 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.647619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.648351 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.648352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.648427 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.648449 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.648499 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.648558 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260926 04:05:37.648629 15581 raft_consensus.cc:374] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:37.648770 15581 raft_consensus.cc:729] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990554ad803a4e10b0a1b3a48450bbe5, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.648921 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.645156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45553 (local address 127.0.200.130:52785)
0926 04:05:37.645332 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.645341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.645382 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:37.645598 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:37.645599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:37.645600 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:37.645657 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.645658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.646600 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.646601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.647824 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:37.647827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.648615 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.648616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:37.648689 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.648694 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:05:37.648793 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 04:05:37.648834 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260926 04:05:37.648919 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.645377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:52785 (local address 127.0.200.129:45553)
0926 04:05:37.645426 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.645428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.645439 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:05:37.645469 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.645470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.645494 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.645597 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:37.645741 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.646483 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.647917 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.648526 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.648590 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.648657 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.648804 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.648853 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:05:37.648917 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.645380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:34267 (local address 127.0.200.131:33959)
0926 04:05:37.645724 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.645726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.645734 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:37.645754 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.645754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.645763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.645856 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:37.645979 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.646506 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.647728 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.648244 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.648312 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.648320 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.648827 (+ 507us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.648863 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323}
I20260926 04:05:37.648926 15581 consensus_queue.cc:260] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.649164 15251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272cef9906a6474c85bb049cfddf89d6" candidate_uuid: "28bfda48fc5e4745a76f92a913a243c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c976e0eb7654e90af9416b904d1bed1" is_pre_election: true
I20260926 04:05:37.649230 15526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272cef9906a6474c85bb049cfddf89d6" candidate_uuid: "28bfda48fc5e4745a76f92a913a243c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990554ad803a4e10b0a1b3a48450bbe5" is_pre_election: true
I20260926 04:05:37.649331 15526 raft_consensus.cc:2454] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bfda48fc5e4745a76f92a913a243c0 in term 0.
I20260926 04:05:37.649333 15251 raft_consensus.cc:2454] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bfda48fc5e4745a76f92a913a243c0 in term 0.
I20260926 04:05:37.649506 15581 ts_tablet_manager.cc:1428] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:05:37.649518 15572 heartbeater.cc:503] Master 127.0.200.190:40447 was elected leader, sending a full tablet report...
I20260926 04:05:37.649595 15324 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5; no voters:
I20260926 04:05:37.649726 15440 raft_consensus.cc:2781] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:05:37.649770 15440 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:05:37.649827 15440 raft_consensus.cc:3037] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:37.650736 15440 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.650880 15440 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [CANDIDATE]: Term 1 election: Requested vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553)
I20260926 04:05:37.651081 15526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272cef9906a6474c85bb049cfddf89d6" candidate_uuid: "28bfda48fc5e4745a76f92a913a243c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "990554ad803a4e10b0a1b3a48450bbe5"
I20260926 04:05:37.651189 15526 raft_consensus.cc:3037] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:37.651788 15526 raft_consensus.cc:2454] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bfda48fc5e4745a76f92a913a243c0 in term 1.
I20260926 04:05:37.651911 15251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "272cef9906a6474c85bb049cfddf89d6" candidate_uuid: "28bfda48fc5e4745a76f92a913a243c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c976e0eb7654e90af9416b904d1bed1"
I20260926 04:05:37.651990 15324 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5; no voters:
I20260926 04:05:37.651994 15251 raft_consensus.cc:3037] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:37.652103 15440 raft_consensus.cc:2781] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:37.652293 15440 raft_consensus.cc:686] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 1 LEADER]: Becoming Leader. State: Replica: 28bfda48fc5e4745a76f92a913a243c0, State: Running, Role: LEADER
I20260926 04:05:37.652390 15440 consensus_queue.cc:237] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.652786 15251 raft_consensus.cc:2454] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bfda48fc5e4745a76f92a913a243c0 in term 1.
I20260926 04:05:37.653080 15108 catalog_manager.cc:5495] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130). New cstate: current_term: 1 leader_uuid: "28bfda48fc5e4745a76f92a913a243c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260926 04:05:37.658121 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.656176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.0.200.130:44377)
0926 04:05:37.656226 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:37.656227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:37.656233 (+ 6us) server_negotiation.cc:407] Connection header received
0926 04:05:37.656258 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:37.656258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:37.656272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:37.656330 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:05:37.656435 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.656897 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.657556 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:37.657747 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:37.657796 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:37.657902 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:05:37.657981 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:05:37.658026 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:37.658061 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 04:05:37.658075 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 04:05:37.659518 15251 raft_consensus.cc:1261] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Refusing update from remote peer 28bfda48fc5e4745a76f92a913a243c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:37.659518 15526 raft_consensus.cc:1261] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Refusing update from remote peer 28bfda48fc5e4745a76f92a913a243c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:37.659866 15585 consensus_queue.cc:1035] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:37.660037 15440 consensus_queue.cc:1035] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:37.661934 15588 mvcc.cc:205] Tried to move back new op lower bound from 7333460122250858496 to 7333460122224746496. Current Snapshot: MvccSnapshot[applied={T|T < 7333460122250858496}]
I20260926 04:05:37.662529 15590 mvcc.cc:205] Tried to move back new op lower bound from 7333460122250858496 to 7333460122224746496. Current Snapshot: MvccSnapshot[applied={T|T < 7333460122250858496}]
I20260926 04:05:37.664067 15109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:37.664152 15109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:37.664835 15109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Sending DeleteTablet for 3 replicas of tablet 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:37.665102 15506 tablet_service.cc:1514] Processing DeleteTablet for tablet 272cef9906a6474c85bb049cfddf89d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:44150
I20260926 04:05:37.665131 15370 tablet_service.cc:1514] Processing DeleteTablet for tablet 272cef9906a6474c85bb049cfddf89d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:34036
I20260926 04:05:37.665160 15231 tablet_service.cc:1514] Processing DeleteTablet for tablet 272cef9906a6474c85bb049cfddf89d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:37 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260926 04:05:37.665223 15443 tablet_replica.cc:330] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: stopping tablet replica
I20260926 04:05:37.665234 15306 tablet_replica.cc:330] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: stopping tablet replica
I20260926 04:05:37.665269 15443 raft_consensus.cc:2229] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:05:37.665270 15306 raft_consensus.cc:2229] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.665330 15306 raft_consensus.cc:2258] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.665344 15443 raft_consensus.cc:2258] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.665596 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15147
I20260926 04:05:37.665625 15597 tablet_replica.cc:330] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: stopping tablet replica
I20260926 04:05:37.665714 15597 raft_consensus.cc:2229] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:37.665727 15306 ts_tablet_manager.cc:1905] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:37.665805 15597 raft_consensus.cc:2258] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:37.666839 15597 ts_tablet_manager.cc:1905] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:37.668581 15443 ts_tablet_manager.cc:1905] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:37.668655 15597 ts_tablet_manager.cc:1918] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:37.668722 15597 log.cc:1196] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:37.668978 15597 ts_tablet_manager.cc:1939] T 272cef9906a6474c85bb049cfddf89d6 P 990554ad803a4e10b0a1b3a48450bbe5: Deleting consensus metadata
I20260926 04:05:37.669391 15095 catalog_manager.cc:4841] TS 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): tablet 272cef9906a6474c85bb049cfddf89d6 (table test-table [id=133845a0cd244ebabb5c497b0892edb2]) successfully deleted
I20260926 04:05:37.670028 15443 ts_tablet_manager.cc:1918] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:37.670090 15443 log.cc:1196] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:37.670315 15443 ts_tablet_manager.cc:1939] T 272cef9906a6474c85bb049cfddf89d6 P 28bfda48fc5e4745a76f92a913a243c0: Deleting consensus metadata
I20260926 04:05:37.670655 15096 catalog_manager.cc:4841] TS 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): tablet 272cef9906a6474c85bb049cfddf89d6 (table test-table [id=133845a0cd244ebabb5c497b0892edb2]) successfully deleted
W20260926 04:05:37.674677 15094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:45553 (error 108)
I20260926 04:05:37.675051 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15301
I20260926 04:05:37.675220 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:37.675056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:45553 (local address 127.0.0.1:49336)
0926 04:05:37.675129 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.675187 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 04:05:37.676271 15094 catalog_manager.cc:4584] TS 9c976e0eb7654e90af9416b904d1bed1 (127.0.200.129:45553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 272cef9906a6474c85bb049cfddf89d6: Network error: Client connection negotiation failed: client connection to 127.0.200.129:45553: connect: Connection refused (error 111)
I20260926 04:05:37.679947 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15441
I20260926 04:05:37.684638 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15076
I20260926 04:05:37.689745 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--webserver_interface=127.0.200.190
--webserver_port=36559
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:40447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:37.774441 15599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.774744 15599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.774789 15599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.776096 15599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:37.776171 15599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.776198 15599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.776228 15599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:37.776264 15599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:37.777897 15599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:40447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=36559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.778208 15599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.778465 15599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.783962 15604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.783962 15607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.783995 15605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.784142 15599 server_base.cc:1034] running on GCE node
I20260926 04:05:37.784435 15599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.784759 15599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.785931 15599 hybrid_clock.cc:648] HybridClock initialized: now 1790395537785917 us; error 29 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.788992 15599 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:37.789574 15599 webserver.cc:460] Webserver started at http://127.0.200.190:36559/ using document root <none> and password file <none>
I20260926 04:05:37.789835 15599 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.789912 15599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.791546 15599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:37.792366 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.792572 15599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:37.792682 15599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "43b86c5bbb794f51a4a7700ac034c787"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "9c84bcfaeddeb8c9bc47d720d8fadc78"
server_key_iv: "fa121bf8da719cb7ac1e8fefe82ce1e3"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.793064 15599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.802671 15599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.805023 15599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.805204 15599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.809653 15599 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:40447
I20260926 04:05:37.809795 15666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:40447 every 8 connection(s)
I20260926 04:05:37.810283 15599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:37.812001 15667 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:05:37.812701 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap starting.
I20260926 04:05:37.814716 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15599
I20260926 04:05:37.815018 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:45553
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=35975
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:37.821144 15667 log.cc:824] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.823758 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:37.824213 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787: Bootstrap complete.
I20260926 04:05:37.826447 15667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.826747 15667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.826912 15667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.826978 15667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:37.827040 15667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:37.827083 15667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 2 FOLLOWER]: Advancing to term 3
I20260926 04:05:37.828153 15667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.828248 15667 leader_election.cc:302] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b86c5bbb794f51a4a7700ac034c787; no voters:
I20260926 04:05:37.828469 15667 leader_election.cc:288] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 04:05:37.828516 15671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 04:05:37.828642 15671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [term 3 LEADER]: Becoming Leader. State: Replica: 43b86c5bbb794f51a4a7700ac034c787, State: Running, Role: LEADER
I20260926 04:05:37.828748 15667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:37.828737 15671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } }
I20260926 04:05:37.829000 15673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:37.829085 15673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:37.828995 15671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b86c5bbb794f51a4a7700ac034c787. Latest consensus state: current_term: 3 leader_uuid: "43b86c5bbb794f51a4a7700ac034c787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b86c5bbb794f51a4a7700ac034c787" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40447 } } }
I20260926 04:05:37.829272 15671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b86c5bbb794f51a4a7700ac034c787 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:37.829393 15676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:37.830240 15676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:37.830348 15676 catalog_manager.cc:1259] Loaded cluster ID: b25a32d537bf44c494bea40f9d623fd3
I20260926 04:05:37.830384 15676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 04:05:37.830621 15676 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb9a78709ad google::LogMessage::Fail() at ??:0
@ 0x7fb9a787388a google::LogMessage::SendToLog() at ??:0
@ 0x7fb9a78704de google::LogMessage::Flush() at ??:0
@ 0x7fb9a7871a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb9af7950f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb9af79e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb9a973e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb9a9733954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb9aa5976db start_thread at ??:0
@ 0x7fb9a636e61f clone at ??:0
W20260926 04:05:37.931401 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:37.931597 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:37.931622 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:37.932796 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:37.932837 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:37.932863 15669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:37.934326 15669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:45553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=35975
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:37.934592 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:37.934820 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:37.939982 15706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.940145 15669 server_base.cc:1034] running on GCE node
W20260926 04:05:37.939961 15703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:37.939961 15704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:37.940531 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:37.940869 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:37.942047 15669 hybrid_clock.cc:648] HybridClock initialized: now 1790395537942021 us; error 42 us; skew 500 ppm
Sep 26 04:05:37 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:37.945039 15669 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:37.945544 15669 webserver.cc:460] Webserver started at http://127.0.200.129:35975/ using document root <none> and password file <none>
I20260926 04:05:37.945850 15669 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:37.945928 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:37.947573 15669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.948469 15713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:37.948683 15669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:37.948796 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "9c976e0eb7654e90af9416b904d1bed1"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "6a9de0972a80b782b4acdfa9bedbaf6a"
server_key_iv: "ebe32fb76f82762cafb4a95f5dc6b1c6"
server_key_version: "encryptionkey@0"
I20260926 04:05:37.949152 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:37.961830 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:37.964581 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:37.964794 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:37.965077 15669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:37.965603 15720 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 04:05:37.967628 15669 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 04:05:37.967717 15669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 04:05:37.967767 15669 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 04:05:37.968494 15669 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 04:05:37.968556 15669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 04:05:37.968636 15720 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap starting.
I20260926 04:05:37.970799 15720 log.cc:824] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:37.973786 15720 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:05:37.974107 15720 tablet_bootstrap.cc:492] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Bootstrap complete.
I20260926 04:05:37.974269 15720 ts_tablet_manager.cc:1397] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260926 04:05:37.975781 15669 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:45553
I20260926 04:05:37.975884 15828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:45553 every 8 connection(s)
I20260926 04:05:37.976161 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:05:37.976190 15720 raft_consensus.cc:348] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.977133 15720 raft_consensus.cc:729] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c976e0eb7654e90af9416b904d1bed1, State: Initialized, Role: FOLLOWER
I20260926 04:05:37.977291 15720 consensus_queue.cc:260] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:37.977514 15720 ts_tablet_manager.cc:1428] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1: Time spent starting tablet: real 0.003s user 0.004s sys 0.001s
I20260926 04:05:37.980978 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15669
I20260926 04:05:37.981258 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:44377
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=34541
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:38.062667 15831 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:05:37.976863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:40419)
0926 04:05:37.977047 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:37.977096 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:37.977258 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:38.062389 (+ 85131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
W20260926 04:05:38.062984 15829 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:40447 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:40447: Client connection negotiation failed: client connection to 127.0.200.190:40447: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 04:05:38.063474 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.063341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:59889)
0926 04:05:38.063399 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.063440 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 04:05:38.063896 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.063767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:37909)
0926 04:05:38.063843 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.063874 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 04:05:38.064051 15829 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:05:38.072985 15833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:38.073191 15833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:38.073215 15833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:38.074312 15833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:38.074349 15833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:38.074374 15833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:38.075863 15833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=34541
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:38.076045 15833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:38.076228 15833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:05:38.081521 15833 server_base.cc:1034] running on GCE node
W20260926 04:05:38.081501 15839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:38.081501 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:38.081490 15841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:38.082000 15833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:38.082231 15833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:38.083419 15833 hybrid_clock.cc:648] HybridClock initialized: now 1790395538083405 us; error 28 us; skew 500 ppm
Sep 26 04:05:38 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:38.086066 15833 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:38.086603 15833 webserver.cc:460] Webserver started at http://127.0.200.130:34541/ using document root <none> and password file <none>
I20260926 04:05:38.086858 15833 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:38.086928 15833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:38.088410 15833 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:38.089165 15848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.089319 15833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:38.089430 15833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "28bfda48fc5e4745a76f92a913a243c0"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "4deebfcd966a6a953385c068094e8460"
server_key_iv: "0c1d80e2242c21fa3c36179a3a723d34"
server_key_version: "encryptionkey@0"
I20260926 04:05:38.089816 15833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:38.106268 15833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:38.108713 15833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:38.108894 15833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:38.109138 15833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:38.109447 15833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:38.109499 15833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.109541 15833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:38.109572 15833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.115417 15833 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:44377
I20260926 04:05:38.115461 15961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:44377 every 8 connection(s)
I20260926 04:05:38.115844 15833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:05:38.116878 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.116500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:49633)
0926 04:05:38.116673 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.116742 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260926 04:05:38.117153 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15833
W20260926 04:05:38.117364 15962 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:40447 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:40447: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
I20260926 04:05:38.117436 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:33959
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:40447
--webserver_port=34811
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:05:38.117645 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.117550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:42719)
0926 04:05:38.117593 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.117616 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 04:05:38.118026 15964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.117917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:57103)
0926 04:05:38.117968 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.117997 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 04:05:38.118172 15962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:05:38.199501 15965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:38.199712 15965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:38.199743 15965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:38.200919 15965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:38.200966 15965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:38.200994 15965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:38.202572 15965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:33959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=34811
--tserver_master_addrs=127.0.200.190:40447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:38.202829 15965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:38.203096 15965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:38.208277 15971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:38.208316 15970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:38.208461 15965 server_base.cc:1034] running on GCE node
W20260926 04:05:38.208509 15973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:38.208755 15965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:38.209043 15965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:38.210222 15965 hybrid_clock.cc:648] HybridClock initialized: now 1790395538210210 us; error 45 us; skew 500 ppm
Sep 26 04:05:38 dist-test-slave-j8c1 krb5kdc[14527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:38.213124 15965 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:38.213662 15965 webserver.cc:460] Webserver started at http://127.0.200.131:34811/ using document root <none> and password file <none>
I20260926 04:05:38.213922 15965 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:38.213995 15965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:38.215502 15965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:38.216204 15980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.216534 15965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:38.216651 15965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "990554ad803a4e10b0a1b3a48450bbe5"
format_stamp: "Formatted at 2026-09-26 04:05:36 on dist-test-slave-j8c1"
server_key: "5bace68786c9ad19de41d21bd67509ff"
server_key_iv: "20e87dac2c0dd7bf28b8293c7cdc2278"
server_key_version: "encryptionkey@0"
I20260926 04:05:38.217032 15965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:38.224148 15965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:38.226032 15965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:38.226197 15965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:38.226421 15965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:38.226815 15965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:38.226867 15965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.226895 15965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:38.226913 15965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:38.232976 15965 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:33959
I20260926 04:05:38.233023 16093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:33959 every 8 connection(s)
I20260926 04:05:38.233371 15965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:05:38.234408 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.234022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:46967)
0926 04:05:38.234189 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.234265 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
W20260926 04:05:38.234858 16094 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:40447 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:40447: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
I20260926 04:05:38.235162 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.235071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:56351)
0926 04:05:38.235118 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.235144 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 04:05:38.235466 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:38.235394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:51071)
0926 04:05:38.235431 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:38.235451 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 04:05:38.235579 16094 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 04:05:38.242758 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 15965
I20260926 04:05:39.065094 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.064725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:36963)
0926 04:05:39.064939 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:39.064993 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260926 04:05:39.118851 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.118546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:57933)
0926 04:05:39.118758 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:39.118793 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260926 04:05:39.236246 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.235995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:34333)
0926 04:05:39.236147 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:39.236186 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260926 04:05:39.338160 16101 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:39.338274 16101 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:39.338637 16101 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
Sep 26 04:05:39 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.130@KRBTEST.COM
Sep 26 04:05:39 dist-test-slave-j8c1 krb5kdc[14527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.131@KRBTEST.COM
I20260926 04:05:39.349260 16103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.338819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:39757 (local address 127.0.200.131:33959)
0926 04:05:39.338983 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:39.338995 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:39.339032 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:05:39.339545 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:39.339547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:39.339591 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:39.339741 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:39.340545 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.341371 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342339 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.342779 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342921 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:39.345683 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:39.345686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:39.345687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:39.345705 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:39.347700 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:39.348139 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:39.348141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:39.348142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:39.348237 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:39.348457 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:39.348458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:39.348458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:39.348692 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:39.348772 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:39.349001 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 04:05:39.349126 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260926 04:05:39.349261 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.338967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:49301 (local address 127.0.200.130:44377)
0926 04:05:39.339121 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:39.339128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:39.339154 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:05:39.339563 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:39.339565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:39.339600 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:39.339740 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:39.340444 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.341307 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342321 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.342778 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342890 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:39.345322 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:39.345357 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:39.345358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:39.345370 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:39.347463 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:39.347948 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:39.347951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:39.347952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:39.348063 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:39.348334 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:39.348335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:39.348335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:39.348618 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:39.348684 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:39.349027 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 04:05:39.349149 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260926 04:05:39.349260 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.338718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:33959 (local address 127.0.200.129:39757)
0926 04:05:39.338770 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:39.338784 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:39.338844 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:39.339776 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:39.339779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:39.340361 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:39.340415 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:39.340417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.341471 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.341472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:39.342234 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:39.342236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.342888 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:39.343011 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:39.343610 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:39.343618 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:39.345564 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:39.347833 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:39.347834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:39.347834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:39.348053 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:39.348327 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:39.348328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:39.348329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:39.348394 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:39.348774 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:39.348774 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:39.348843 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 04:05:39.348925 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 04:05:39.349140 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":50}
I20260926 04:05:39.349361 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:39.338765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:44377 (local address 127.0.200.129:49301)
0926 04:05:39.338894 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:39.338905 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:39.338963 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:39.339777 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:39.339779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:39.340126 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:39.340319 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:39.340324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.341428 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.341429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:39.342231 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:39.342233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:39.342882 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:39.342883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:39.343010 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:39.343670 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:39.343675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:39.345189 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:39.347555 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:39.347562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:39.347564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:39.347860 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:39.348131 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:39.348132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:39.348132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:39.348260 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:39.348695 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:39.348701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:39.348844 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 04:05:39.348955 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 04:05:39.349221 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":137,"thread_start_us":58,"threads_started":1}
W20260926 04:05:39.350622 15716 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
W20260926 04:05:39.350698 15717 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:39.350760 15716 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c976e0eb7654e90af9416b904d1bed1; no voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5
I20260926 04:05:39.350867 16101 raft_consensus.cc:2726] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:05:40.066021 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:40.065740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:48757)
0926 04:05:40.065908 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:40.065965 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260926 04:05:40.119653 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:40.119362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:41955)
0926 04:05:40.119560 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:40.119606 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":64,"threads_started":1}
I20260926 04:05:40.236986 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:40.236749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:51417)
0926 04:05:40.236900 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:40.236944 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260926 04:05:41.067031 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:41.066763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:56687)
0926 04:05:41.066926 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:41.066982 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260926 04:05:41.120426 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:41.120188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:44619)
0926 04:05:41.120334 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:41.120377 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260926 04:05:41.237897 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:41.237681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:45243)
0926 04:05:41.237815 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:41.237854 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260926 04:05:41.312142 16111 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:41.312235 16111 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:41.312495 16111 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
W20260926 04:05:41.313001 15717 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
W20260926 04:05:41.313071 15716 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:41.313099 15717 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c976e0eb7654e90af9416b904d1bed1; no voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5
I20260926 04:05:41.313167 16111 raft_consensus.cc:2726] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:05:42.068089 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:42.067724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:34195)
0926 04:05:42.067932 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:42.068004 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":65,"threads_started":1}
I20260926 04:05:42.121122 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:42.120908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:45427)
0926 04:05:42.121044 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:42.121078 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260926 04:05:42.238698 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:42.238409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:57705)
0926 04:05:42.238529 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:42.238646 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260926 04:05:43.069132 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:43.068839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:42059)
0926 04:05:43.069034 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:43.069082 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
I20260926 04:05:43.122164 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:43.121821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:40013)
0926 04:05:43.122035 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:43.122103 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
I20260926 04:05:43.239545 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:43.239292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:43915)
0926 04:05:43.239447 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:43.239497 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260926 04:05:43.477734 16118 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:43.477845 16118 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:43.478041 16118 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
W20260926 04:05:43.478549 15717 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
W20260926 04:05:43.478662 15716 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:43.478755 15717 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c976e0eb7654e90af9416b904d1bed1; no voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5
I20260926 04:05:43.478878 16118 raft_consensus.cc:2726] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:05:44.070201 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:44.069942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:39829)
0926 04:05:44.070112 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:44.070152 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260926 04:05:44.123183 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:44.122937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:44773)
0926 04:05:44.123087 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:44.123135 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260926 04:05:44.240348 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:44.240078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:60185)
0926 04:05:44.240234 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:44.240289 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260926 04:05:45.071245 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:45.071007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:37419)
0926 04:05:45.071157 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:45.071197 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260926 04:05:45.123996 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:45.123733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:43727)
0926 04:05:45.123905 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:45.123948 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260926 04:05:45.241204 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:45.240957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:38751)
0926 04:05:45.241116 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:45.241157 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260926 04:05:46.047768 16125 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:46.047987 16125 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:46.048182 16125 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
W20260926 04:05:46.048609 15716 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
W20260926 04:05:46.048679 15717 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:46.048784 15717 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c976e0eb7654e90af9416b904d1bed1; no voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5
I20260926 04:05:46.048852 16125 raft_consensus.cc:2726] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:05:46.071997 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:46.071757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:38101)
0926 04:05:46.071901 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:46.071943 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":58,"threads_started":1}
I20260926 04:05:46.124944 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:46.124706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:44477)
0926 04:05:46.124834 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:46.124879 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260926 04:05:46.242141 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:46.241917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:48807)
0926 04:05:46.242047 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:46.242095 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260926 04:05:47.072811 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:47.072580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:51915)
0926 04:05:47.072717 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:47.072762 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260926 04:05:47.125799 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:47.125568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:53713)
0926 04:05:47.125709 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:47.125753 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260926 04:05:47.242973 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:47.242731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:40129)
0926 04:05:47.242886 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:47.242925 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260926 04:05:48.073612 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:48.073377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:46873)
0926 04:05:48.073518 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:48.073563 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260926 04:05:48.126677 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:48.126372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:53511)
0926 04:05:48.126524 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:48.126628 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260926 04:05:48.243803 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:48.243560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:42327)
0926 04:05:48.243711 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:48.243757 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260926 04:05:49.074519 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:49.074237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:55367)
0926 04:05:49.074430 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:49.074472 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260926 04:05:49.127492 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:49.127204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:50309)
0926 04:05:49.127362 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:49.127442 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260926 04:05:49.244616 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:49.244346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:36659)
0926 04:05:49.244512 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:49.244558 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260926 04:05:50.075395 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:50.075151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:47645)
0926 04:05:50.075302 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:50.075349 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260926 04:05:50.128207 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:50.127990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:60845)
0926 04:05:50.128115 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:50.128159 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260926 04:05:50.245527 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:50.245226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:48859)
0926 04:05:50.245420 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:50.245469 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260926 04:05:51.076540 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:51.076244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:53977)
0926 04:05:51.076445 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:51.076490 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260926 04:05:51.129040 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:51.128796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:33095)
0926 04:05:51.128941 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:51.128988 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260926 04:05:51.246408 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:51.246153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:56501)
0926 04:05:51.246316 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:51.246361 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260926 04:05:52.077697 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:52.077490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:41875)
0926 04:05:52.077611 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:52.077648 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260926 04:05:52.129845 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:52.129601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:46081)
0926 04:05:52.129762 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:52.129800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260926 04:05:52.247175 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:52.246941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:45905)
0926 04:05:52.247085 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:52.247128 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260926 04:05:52.314927 16147 raft_consensus.cc:482] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:52.315048 16147 raft_consensus.cc:504] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "990554ad803a4e10b0a1b3a48450bbe5" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 33959 } } peers { permanent_uuid: "28bfda48fc5e4745a76f92a913a243c0" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 44377 } } peers { permanent_uuid: "9c976e0eb7654e90af9416b904d1bed1" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 45553 } }
I20260926 04:05:52.315297 16147 leader_election.cc:288] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959), 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377)
W20260926 04:05:52.315665 15716 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 990554ad803a4e10b0a1b3a48450bbe5 (127.0.200.131:33959): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
W20260926 04:05:52.315768 15717 leader_election.cc:341] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bfda48fc5e4745a76f92a913a243c0 (127.0.200.130:44377): Not found: Tablet not found: 272cef9906a6474c85bb049cfddf89d6
I20260926 04:05:52.315833 15717 leader_election.cc:302] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c976e0eb7654e90af9416b904d1bed1; no voters: 28bfda48fc5e4745a76f92a913a243c0, 990554ad803a4e10b0a1b3a48450bbe5
I20260926 04:05:52.315898 16147 raft_consensus.cc:2726] T 272cef9906a6474c85bb049cfddf89d6 P 9c976e0eb7654e90af9416b904d1bed1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:05:53.078517 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:53.078264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.129:54429)
0926 04:05:53.078426 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:53.078471 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
I20260926 04:05:53.130738 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:53.130470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.130:39739)
0926 04:05:53.130650 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:53.130692 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":168,"threads_started":1}
I20260926 04:05:53.247963 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:53.247752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:40447 (local address 127.0.200.131:34187)
0926 04:05:53.247878 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:53.247918 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:40447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260926 04:05:53.249284 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15669
I20260926 04:05:53.254137 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15833
I20260926 04:05:53.258873 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 15965
2026-09-26T04:05:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18454 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:05:53 dist-test-slave-j8c1 krb5kdc[16153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:05:53 dist-test-slave-j8c1 krb5kdc[16153](info): set up 2 sockets
Sep 26 04:05:53 dist-test-slave-j8c1 krb5kdc[16153](info): commencing operation
krb5kdc: starting...
W20260926 04:05:54.314827 802 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:05:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:05:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:05:54.437563 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:54.519260 16166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:54.519522 16166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:54.519563 16166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:54.520891 16166 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 04:05:54.520972 16166 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 04:05:54.520996 16166 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 04:05:54.521014 16166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:05:54.521044 16166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:54.521073 16166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:54.521096 16166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:05:54.521124 16166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:05:54.522768 16166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:54.523079 16166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:54.523313 16166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:54.528450 16174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.528463 16171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.528450 16172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:54.528716 16166 server_base.cc:1034] running on GCE node
I20260926 04:05:54.528966 16166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:54.529250 16166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:54.530400 16166 hybrid_clock.cc:648] HybridClock initialized: now 1790395554530389 us; error 31 us; skew 500 ppm
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:54.533003 16166 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:05:54.533427 16166 webserver.cc:460] Webserver started at http://127.0.200.190:46593/ using document root <none> and password file <none>
I20260926 04:05:54.533658 16166 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:54.533752 16166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:54.533890 16166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:54.535068 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "54d03d80711a4a93a856f8ab0b1e7c87"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "8b60722ad9932cc55c6010893d97ebd9"
server_key_iv: "df91c7a023e094d90e76c752f206312d"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.535420 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "54d03d80711a4a93a856f8ab0b1e7c87"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "8b60722ad9932cc55c6010893d97ebd9"
server_key_iv: "df91c7a023e094d90e76c752f206312d"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.536741 16166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:05:54.537520 16181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.537707 16166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:05:54.537817 16166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "54d03d80711a4a93a856f8ab0b1e7c87"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "8b60722ad9932cc55c6010893d97ebd9"
server_key_iv: "df91c7a023e094d90e76c752f206312d"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.537892 16166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:54.552382 16166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:54.554838 16166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:54.555017 16166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:54.559127 16166 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:39941
I20260926 04:05:54.559165 16233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:39941 every 8 connection(s)
I20260926 04:05:54.559535 16166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:05:54.560395 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:54.562450 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16166
I20260926 04:05:54.562580 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:05:54.562796 802 external_mini_cluster.cc:1428] Setting key a14a5800f3b906ef764a3aa317bdc1f3
I20260926 04:05:54.563025 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap starting.
I20260926 04:05:54.563735 16234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:54.564045 16234 log.cc:824] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:54.564749 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: No bootstrap required, opened a new log
I20260926 04:05:54.566632 16234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:05:54.566793 16234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:54.566839 16234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Initialized, Role: FOLLOWER
I20260926 04:05:54.566970 16234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:05:54.567060 16234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:05:54.567103 16234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:05:54.567149 16234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:54.567713 16234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:05:54.567832 16234 leader_election.cc:302] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d03d80711a4a93a856f8ab0b1e7c87; no voters:
I20260926 04:05:54.568043 16234 leader_election.cc:288] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:05:54.568079 16239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:54.568199 16239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 LEADER]: Becoming Leader. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Running, Role: LEADER
I20260926 04:05:54.568344 16239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:05:54.568348 16234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:05:54.569659 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:05:54.569767 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:54.569973 16254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:05:54.570128 16240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d03d80711a4a93a856f8ab0b1e7c87. Latest consensus state: current_term: 1 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:05:54.570248 16240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
I20260926 04:05:54.570675 16254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:05:54.572633 16254 catalog_manager.cc:1347] Generated new cluster ID: e9f9404d50784b87b11e2273d5712a3b
I20260926 04:05:54.572687 16254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:05:54.575456 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:54.563323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.0.200.190:39941)
0926 04:05:54.563544 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:54.563552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:54.563578 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:05:54.564031 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:54.564038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:54.564101 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:54.564258 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:54.564978 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.565868 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.566698 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.567259 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.567400 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:54.570459 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:54.570463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:54.570463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:54.570476 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:54.572951 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.573570 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.573574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.573574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.573675 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.573982 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.573983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.573983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.574284 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:54.574416 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:54.574946 (+ 530us) server_negotiation.cc:299] Negotiation successful
0926 04:05:54.575080 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260926 04:05:54.596889 16254 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:05:54.597460 16254 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:05:54.602620 16254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Generated new TSK 0
I20260926 04:05:54.602823 16254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:05:54.639873 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:54.744406 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:54.744637 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:54.744661 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:54.745851 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:54.745896 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:54.745924 16260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:05:54.747406 16260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:54.747658 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:54.747900 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:54.752925 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.752915 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.752915 16266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:54.753159 16260 server_base.cc:1034] running on GCE node
I20260926 04:05:54.753358 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:54.753585 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:54.754745 16260 hybrid_clock.cc:648] HybridClock initialized: now 1790395554754734 us; error 31 us; skew 500 ppm
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:54.757448 16260 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:05:54.757892 16260 webserver.cc:460] Webserver started at http://127.0.200.129:39987/ using document root <none> and password file <none>
I20260926 04:05:54.758132 16260 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:54.758199 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:54.758334 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:54.759444 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "b3eff65f5e3343479049af6691b2dcb6"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "6855ae92c2a904c17f2070ea7f26c84e"
server_key_iv: "8e9b2854c07a243bcf06a7bbab90e489"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.759799 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "b3eff65f5e3343479049af6691b2dcb6"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "6855ae92c2a904c17f2070ea7f26c84e"
server_key_iv: "8e9b2854c07a243bcf06a7bbab90e489"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.761111 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:05:54.761852 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.762051 16260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:05:54.762152 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b3eff65f5e3343479049af6691b2dcb6"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "6855ae92c2a904c17f2070ea7f26c84e"
server_key_iv: "8e9b2854c07a243bcf06a7bbab90e489"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.762235 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:54.796089 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:54.798321 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:54.798492 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:54.798847 16260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:54.799296 16260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:54.799336 16260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.799360 16260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:54.799405 16260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.805398 16260 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:44489
I20260926 04:05:54.805441 16388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:44489 every 8 connection(s)
I20260926 04:05:54.805771 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:05:54.806188 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16260
I20260926 04:05:54.806298 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:05:54.806531 802 external_mini_cluster.cc:1428] Setting key 427f84b8e8832eeb550a5ac0550ce264
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:05:54.816443 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:54.806532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44361 (local address 127.0.200.190:39941)
0926 04:05:54.806617 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:54.806620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:54.807101 (+ 481us) server_negotiation.cc:407] Connection header received
0926 04:05:54.807558 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:54.807558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:54.807567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:54.807625 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:54.809038 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.809557 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.810620 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.810999 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.811090 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:54.813482 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:54.813483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:54.813484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:54.813490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:54.815073 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.815512 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.815513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.815513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.815568 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.815778 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.815779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.815779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.815887 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:54.815938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:54.816287 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 04:05:54.816372 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 04:05:54.816762 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:54.806524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:44361)
0926 04:05:54.806748 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:54.806814 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:54.807458 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:54.807722 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:54.807726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:54.808581 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:54.808908 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:54.808913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.809673 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.809674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:54.810480 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:54.810482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.811097 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.811098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:54.811223 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:54.811796 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:54.811805 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:54.813383 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:54.815150 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:54.815157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:54.815159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:54.815439 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:54.815614 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:54.815615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:54.815615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:54.815727 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:54.815942 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:54.815946 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:54.816065 (+ 119us) client_negotiation.cc:769] Sending connection context
0926 04:05:54.816153 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:05:54.816368 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
I20260926 04:05:54.817322 16389 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:05:54.817440 16389 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:54.817664 16389 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:05:54.818217 16198 ts_manager.cc:194] Registered new tserver with Master: b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489)
I20260926 04:05:54.818888 16198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:44361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:05:54.833024 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:54.912904 16394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:54.913110 16394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:54.913133 16394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:54.914314 16394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:54.914358 16394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:54.914386 16394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:05:54.915844 16394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:54.916082 16394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:54.916309 16394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:54.921200 16399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.921252 16400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:54.921272 16402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:54.921396 16394 server_base.cc:1034] running on GCE node
I20260926 04:05:54.921576 16394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:54.921765 16394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:54.922892 16394 hybrid_clock.cc:648] HybridClock initialized: now 1790395554922874 us; error 34 us; skew 500 ppm
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:54.925351 16394 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:05:54.925806 16394 webserver.cc:460] Webserver started at http://127.0.200.130:41813/ using document root <none> and password file <none>
I20260926 04:05:54.926043 16394 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:54.926110 16394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:54.926268 16394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:54.927388 16394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "d1456123a667b886f42f28a9dac16daf"
server_key_iv: "e555190ead378b51eaf0c308c2a562c2"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.927739 16394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "d1456123a667b886f42f28a9dac16daf"
server_key_iv: "e555190ead378b51eaf0c308c2a562c2"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.929080 16394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:05:54.929785 16409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.929946 16394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:05:54.930042 16394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "d1456123a667b886f42f28a9dac16daf"
server_key_iv: "e555190ead378b51eaf0c308c2a562c2"
server_key_version: "encryptionkey@0"
I20260926 04:05:54.930114 16394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:54.940662 16394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:54.942984 16394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:54.943147 16394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:54.943373 16394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:54.943670 16394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:54.943718 16394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.943754 16394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:54.943783 16394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:54.949549 16394 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:34635
I20260926 04:05:54.949599 16522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:34635 every 8 connection(s)
I20260926 04:05:54.949921 16394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:05:54 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:54.957752 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16394
I20260926 04:05:54.957865 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:05:54.958063 802 external_mini_cluster.cc:1428] Setting key fb6f4b098c4d92acde050283f0eb4785
I20260926 04:05:54.958812 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:54.950614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:41445 (local address 127.0.200.190:39941)
0926 04:05:54.950685 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:54.950687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:54.950922 (+ 235us) server_negotiation.cc:407] Connection header received
0926 04:05:54.951385 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:54.951386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:54.951396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:54.951443 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:54.952346 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.952828 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.953754 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.953930 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.953975 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:54.956127 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:54.956129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:54.956129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:54.956136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:54.957339 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.957747 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.957748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.957748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.957826 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:54.958095 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:54.958096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:54.958096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:54.958241 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:54.958314 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:54.958611 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 04:05:54.958715 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 04:05:54.959028 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:54.950620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:41445)
0926 04:05:54.950795 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:54.950831 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:54.951295 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:54.951501 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:54.951504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:54.952072 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:54.952271 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:54.952275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.952900 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.952901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:54.953673 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:54.953675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:54.953994 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:54.953994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:54.954089 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:54.954626 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:54.954641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:54.956028 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:54.957422 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:54.957428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:54.957431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:54.957644 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:54.957881 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:54.957882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:54.957882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:54.958022 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:54.958305 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:54.958311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:54.958446 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 04:05:54.958584 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 04:05:54.958796 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260926 04:05:54.959493 16523 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:05:54.959610 16523 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:54.959793 16523 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:05:54.960166 16198 ts_manager.cc:194] Registered new tserver with Master: 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635)
I20260926 04:05:54.960579 16198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:41445
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:05:54.983302 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:05:55.062664 16528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:05:55.062911 16528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:05:55.062948 16528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:05:55.064097 16528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:05:55.064162 16528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:05:55.064209 16528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:05:55.065739 16528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:05:55.065994 16528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:05:55.066216 16528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:05:55.071439 16533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:55.071445 16534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:05:55.071463 16536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:05:55.071710 16528 server_base.cc:1034] running on GCE node
I20260926 04:05:55.072009 16528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:05:55.072206 16528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:05:55.073364 16528 hybrid_clock.cc:648] HybridClock initialized: now 1790395555073354 us; error 26 us; skew 500 ppm
Sep 26 04:05:55 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:05:55.076011 16528 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:05:55.076396 16528 webserver.cc:460] Webserver started at http://127.0.200.131:41393/ using document root <none> and password file <none>
I20260926 04:05:55.076589 16528 fs_manager.cc:359] Metadata directory not provided
I20260926 04:05:55.076628 16528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:05:55.076714 16528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:05:55.077716 16528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "09c137d28c984a6e8a6f0cec467b505b"
format_stamp: "Formatted at 2026-09-26 04:05:55 on dist-test-slave-j8c1"
server_key: "9514654093fdc37d50219ef665d0ba22"
server_key_iv: "1f98a7ce7823e19a84fb2fd8ef34b360"
server_key_version: "encryptionkey@0"
I20260926 04:05:55.078111 16528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "09c137d28c984a6e8a6f0cec467b505b"
format_stamp: "Formatted at 2026-09-26 04:05:55 on dist-test-slave-j8c1"
server_key: "9514654093fdc37d50219ef665d0ba22"
server_key_iv: "1f98a7ce7823e19a84fb2fd8ef34b360"
server_key_version: "encryptionkey@0"
I20260926 04:05:55.079452 16528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:05:55.080156 16543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:55.080334 16528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:05:55.080404 16528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "09c137d28c984a6e8a6f0cec467b505b"
format_stamp: "Formatted at 2026-09-26 04:05:55 on dist-test-slave-j8c1"
server_key: "9514654093fdc37d50219ef665d0ba22"
server_key_iv: "1f98a7ce7823e19a84fb2fd8ef34b360"
server_key_version: "encryptionkey@0"
I20260926 04:05:55.080458 16528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:05:55.096755 16528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:05:55.099123 16528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:05:55.099239 16528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:05:55.099429 16528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:05:55.099745 16528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:05:55.099781 16528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:55.099821 16528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:05:55.099848 16528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:05:55.105556 16528 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:35555
I20260926 04:05:55.105631 16656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:35555 every 8 connection(s)
I20260926 04:05:55.105954 16528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:05:55.108299 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16528
I20260926 04:05:55.108371 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:05:55.108565 802 external_mini_cluster.cc:1428] Setting key bf3e4f6ab9d7e9577a0bb4dc4ffa9008
Sep 26 04:05:55 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:05:55.115298 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.106686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:56359 (local address 127.0.200.190:39941)
0926 04:05:55.106745 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.106747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.106963 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:05:55.107482 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.107483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.107496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.107559 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:55.108584 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.109286 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.110296 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.110475 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.110528 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.112690 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:55.112691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:55.112692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:55.112698 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:55.114157 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:55.114510 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:55.114511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:55.114511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:55.114607 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:55.114809 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:55.114809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:55.114810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:55.114912 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:55.114963 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.115161 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.115218 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 04:05:55.115523 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.106685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:56359)
0926 04:05:55.106826 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.106875 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.107377 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.107627 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.107631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.108229 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:05:55.108488 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.108493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.109400 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.109401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.110223 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.110225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.110588 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.110589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.110710 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.111254 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:05:55.111262 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:05:55.112597 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:05:55.114230 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:55.114234 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:55.114236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:55.114450 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:55.114660 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:05:55.114661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:05:55.114661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:05:55.114755 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:05:55.114963 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:05:55.114967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:05:55.115072 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.115151 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.115273 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260926 04:05:55.116007 16657 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:05:55.116084 16657 heartbeater.cc:464] Registering TS with master...
I20260926 04:05:55.116221 16657 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:05:55.116544 16198 ts_manager.cc:194] Registered new tserver with Master: 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
I20260926 04:05:55.116883 16198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:56359
I20260926 04:05:55.120221 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:05:55.127046 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.121446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.0.200.190:39941)
0926 04:05:55.121519 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.121521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.121614 (+ 93us) server_negotiation.cc:407] Connection header received
0926 04:05:55.121653 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.121654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.121662 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.121707 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:05:55.122200 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.122727 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.123394 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.123602 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.123652 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.124690 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:05:55.124691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:05:55.124691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:05:55.124697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:05:55.126031 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:55.126331 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:55.126332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:55.126332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:55.126379 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:05:55.126531 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:05:55.126531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:05:55.126532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:05:55.126732 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:05:55.126815 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.126905 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.126962 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 04:05:55.128502 16198 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:05:55.130187 16198 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:05:55.136688 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.132664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.0.200.130:34635)
0926 04:05:55.132865 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.132871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.132882 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:05:55.132922 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.132924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.132941 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.133020 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:55.133272 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.134286 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.135576 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.136291 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136366 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.136425 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.136567 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.136609 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":125,"threads_started":1}
I20260926 04:05:55.136675 16668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.132548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34635 (local address 127.0.0.1:53952)
0926 04:05:55.132668 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.132713 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.132800 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.133117 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.133120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.133122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:55.133184 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.133186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.134378 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.134380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.135484 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.135487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.136381 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.136458 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.136469 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.136554 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.136606 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260926 04:05:55.137127 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.132967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44489 (local address 127.0.0.1:57566)
0926 04:05:55.133386 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.133396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.133457 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.133627 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.133628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.133630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:55.133676 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.133678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.135060 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.135061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.135904 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.135907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.136830 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.136893 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.136897 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.136993 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.137036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":58,"threads_started":1}
I20260926 04:05:55.137379 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.132785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35555 (local address 127.0.0.1:39494)
0926 04:05:55.132909 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.132917 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.132979 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.133718 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.133719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.133721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:55.133760 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.133761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.135030 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.135031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.136130 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.136133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.137028 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.137029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.137102 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.137106 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.137197 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.137270 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":21,"thread_start_us":87,"threads_started":1}
I20260926 04:05:55.137619 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.133168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.0.200.131:35555)
0926 04:05:55.133561 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.133564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.133572 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:55.133618 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.133620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.133635 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.133708 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:55.133814 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.134922 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136214 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.136906 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136992 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.137044 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.137469 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.137522 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":74,"threads_started":1}
I20260926 04:05:55.137770 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.133100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.0.200.129:44489)
0926 04:05:55.133339 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.133343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.133477 (+ 134us) server_negotiation.cc:407] Connection header received
0926 04:05:55.133521 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.133523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.133539 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.133618 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:55.133774 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.134938 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136004 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.136665 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.136733 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.136793 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.137631 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.137680 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
I20260926 04:05:55.137841 16457 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd84ff103e14219aaae02bca1b13abf (DEFAULT_TABLE table=test-table [id=2aa994dfdd234781a1d2b7fca02f8167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:55.138208 16457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd84ff103e14219aaae02bca1b13abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:55.138190 16591 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd84ff103e14219aaae02bca1b13abf (DEFAULT_TABLE table=test-table [id=2aa994dfdd234781a1d2b7fca02f8167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:55.138474 16591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd84ff103e14219aaae02bca1b13abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:55.140991 16674 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Bootstrap starting.
I20260926 04:05:55.141062 16675 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Bootstrap starting.
I20260926 04:05:55.141726 16674 tablet_bootstrap.cc:654] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:55.141660 16323 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd84ff103e14219aaae02bca1b13abf (DEFAULT_TABLE table=test-table [id=2aa994dfdd234781a1d2b7fca02f8167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:05:55.141855 16675 tablet_bootstrap.cc:654] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:55.141975 16323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd84ff103e14219aaae02bca1b13abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:05:55.142266 16674 log.cc:824] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:55.142421 16675 log.cc:824] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:55.143051 16674 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: No bootstrap required, opened a new log
I20260926 04:05:55.143163 16674 ts_tablet_manager.cc:1397] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:55.143185 16675 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: No bootstrap required, opened a new log
I20260926 04:05:55.143242 16675 ts_tablet_manager.cc:1397] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:55.144768 16675 raft_consensus.cc:348] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.144932 16675 raft_consensus.cc:374] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:55.144994 16675 raft_consensus.cc:729] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c137d28c984a6e8a6f0cec467b505b, State: Initialized, Role: FOLLOWER
I20260926 04:05:55.145129 16675 consensus_queue.cc:260] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.145334 16678 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Bootstrap starting.
I20260926 04:05:55.145285 16674 raft_consensus.cc:348] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.145465 16674 raft_consensus.cc:374] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:55.145538 16674 raft_consensus.cc:729] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a74c0ea83e4e5ca36d2ebbb3bb0acd, State: Initialized, Role: FOLLOWER
I20260926 04:05:55.145622 16675 ts_tablet_manager.cc:1428] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:55.145653 16674 consensus_queue.cc:260] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.145767 16657 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:05:55.145974 16674 ts_tablet_manager.cc:1428] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:05:55.146057 16678 tablet_bootstrap.cc:654] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Neither blocks nor log segments found. Creating new log.
I20260926 04:05:55.146099 16523 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:05:55.146723 16678 log.cc:824] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Log is configured to *not* fsync() on all Append() calls
I20260926 04:05:55.147435 16678 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: No bootstrap required, opened a new log
I20260926 04:05:55.147490 16678 ts_tablet_manager.cc:1397] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:55.148934 16678 raft_consensus.cc:348] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.149065 16678 raft_consensus.cc:374] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:05:55.149086 16678 raft_consensus.cc:729] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eff65f5e3343479049af6691b2dcb6, State: Initialized, Role: FOLLOWER
I20260926 04:05:55.149159 16678 consensus_queue.cc:260] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.149346 16678 ts_tablet_manager.cc:1428] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:05:55.149415 16389 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:05:55.151400 16682 raft_consensus.cc:482] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:05:55.151484 16682 raft_consensus.cc:504] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.151783 16682 leader_election.cc:288] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
I20260926 04:05:55.154729 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.151967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:49055 (local address 127.0.200.130:34635)
0926 04:05:55.152020 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.152022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.152166 (+ 144us) server_negotiation.cc:407] Connection header received
0926 04:05:55.152243 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.152245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.152254 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.152315 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:55.152457 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.152938 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.153865 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.154380 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.154436 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.154443 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.154619 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.154656 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:05:55.154742 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.151909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34635 (local address 127.0.200.129:49055)
0926 04:05:55.152122 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.152132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.152177 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.152319 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.152320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.152322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:55.152392 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.152394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.153029 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.153030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.153778 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.153780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.154450 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.154451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.154530 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.154534 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.154611 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.154650 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260926 04:05:55.154771 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.152067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:34435 (local address 127.0.200.131:35555)
0926 04:05:55.152107 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.152109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.152198 (+ 89us) server_negotiation.cc:407] Connection header received
0926 04:05:55.152290 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.152291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.152301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.152350 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:05:55.152490 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.152946 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.153870 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.154400 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.154482 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.154489 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.154660 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.154697 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:05:55.154825 16683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.152013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35555 (local address 127.0.200.129:34435)
0926 04:05:55.152138 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.152148 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:05:55.152222 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:05:55.152362 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:05:55.152363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:05:55.152364 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:05:55.152426 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.152428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.153038 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.153039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.153783 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:05:55.153784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.154491 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.154492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:05:55.154595 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.154599 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:05:55.154654 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:05:55.154692 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260926 04:05:55.155081 16611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd84ff103e14219aaae02bca1b13abf" candidate_uuid: "b3eff65f5e3343479049af6691b2dcb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c137d28c984a6e8a6f0cec467b505b" is_pre_election: true
I20260926 04:05:55.155097 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd84ff103e14219aaae02bca1b13abf" candidate_uuid: "b3eff65f5e3343479049af6691b2dcb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" is_pre_election: true
I20260926 04:05:55.155225 16477 raft_consensus.cc:2454] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3eff65f5e3343479049af6691b2dcb6 in term 0.
I20260926 04:05:55.155220 16611 raft_consensus.cc:2454] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3eff65f5e3343479049af6691b2dcb6 in term 0.
I20260926 04:05:55.155421 16277 leader_election.cc:302] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a74c0ea83e4e5ca36d2ebbb3bb0acd, b3eff65f5e3343479049af6691b2dcb6; no voters:
I20260926 04:05:55.155563 16682 raft_consensus.cc:2781] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:05:55.155622 16682 raft_consensus.cc:482] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:05:55.155640 16682 raft_consensus.cc:3037] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:55.156306 16682 raft_consensus.cc:504] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.156431 16682 leader_election.cc:288] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 1 election: Requested vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
I20260926 04:05:55.156597 16611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd84ff103e14219aaae02bca1b13abf" candidate_uuid: "b3eff65f5e3343479049af6691b2dcb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09c137d28c984a6e8a6f0cec467b505b"
I20260926 04:05:55.156589 16477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd84ff103e14219aaae02bca1b13abf" candidate_uuid: "b3eff65f5e3343479049af6691b2dcb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
I20260926 04:05:55.156656 16611 raft_consensus.cc:3037] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:55.156675 16477 raft_consensus.cc:3037] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:05:55.157308 16611 raft_consensus.cc:2454] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3eff65f5e3343479049af6691b2dcb6 in term 1.
I20260926 04:05:55.157428 16477 raft_consensus.cc:2454] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3eff65f5e3343479049af6691b2dcb6 in term 1.
I20260926 04:05:55.157446 16277 leader_election.cc:302] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c137d28c984a6e8a6f0cec467b505b, b3eff65f5e3343479049af6691b2dcb6; no voters:
I20260926 04:05:55.157529 16682 raft_consensus.cc:2781] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:05:55.157666 16682 raft_consensus.cc:686] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 LEADER]: Becoming Leader. State: Replica: b3eff65f5e3343479049af6691b2dcb6, State: Running, Role: LEADER
I20260926 04:05:55.157763 16682 consensus_queue.cc:237] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:05:55.158524 16198 catalog_manager.cc:5495] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129). New cstate: current_term: 1 leader_uuid: "b3eff65f5e3343479049af6691b2dcb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } health_report { overall_health: HEALTHY } } }
I20260926 04:05:55.164011 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.161929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.0.200.129:44489)
0926 04:05:55.161980 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:05:55.161982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:05:55.161990 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:05:55.162045 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:05:55.162046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:05:55.162059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:05:55.162112 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:05:55.162209 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.162736 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.163426 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:05:55.163595 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:05:55.163641 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:05:55.163762 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:05:55.163844 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:05:55.163899 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:05:55.163911 (+ 12us) server_negotiation.cc:299] Negotiation successful
0926 04:05:55.163925 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 04:05:55.165688 16477 raft_consensus.cc:1261] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Refusing update from remote peer b3eff65f5e3343479049af6691b2dcb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:55.165736 16611 raft_consensus.cc:1261] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Refusing update from remote peer b3eff65f5e3343479049af6691b2dcb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:05:55.166002 16682 consensus_queue.cc:1035] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:55.166119 16684 consensus_queue.cc:1035] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:05:55.168608 16688 mvcc.cc:205] Tried to move back new op lower bound from 7333460193955872768 to 7333460193926635520. Current Snapshot: MvccSnapshot[applied={T|T < 7333460193955872768}]
I20260926 04:05:55.168965 16687 mvcc.cc:205] Tried to move back new op lower bound from 7333460193955872768 to 7333460193926635520. Current Snapshot: MvccSnapshot[applied={T|T < 7333460193955872768}]
I20260926 04:05:55.171296 16198 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:55.171398 16198 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:05:55.172087 16198 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Sending DeleteTablet for 3 replicas of tablet 0fd84ff103e14219aaae02bca1b13abf
I20260926 04:05:55.172458 16457 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd84ff103e14219aaae02bca1b13abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:55 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:53952
I20260926 04:05:55.172469 16591 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd84ff103e14219aaae02bca1b13abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:55 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:39494
I20260926 04:05:55.172683 16697 tablet_replica.cc:330] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: stopping tablet replica
I20260926 04:05:55.172683 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16260
I20260926 04:05:55.172797 16697 raft_consensus.cc:2229] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:55.172909 16697 raft_consensus.cc:2258] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:55.174044 16697 ts_tablet_manager.cc:1905] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:55.174263 16698 tablet_replica.cc:330] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: stopping tablet replica
I20260926 04:05:55.174371 16698 raft_consensus.cc:2229] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:05:55.174490 16698 raft_consensus.cc:2258] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:05:55.175055 16698 ts_tablet_manager.cc:1905] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:05:55.176131 16697 ts_tablet_manager.cc:1918] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:55.176249 16697 log.cc:1196] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/0fd84ff103e14219aaae02bca1b13abf
I20260926 04:05:55.176607 16697 ts_tablet_manager.cc:1939] T 0fd84ff103e14219aaae02bca1b13abf P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting consensus metadata
I20260926 04:05:55.176684 16698 ts_tablet_manager.cc:1918] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:05:55.176751 16698 log.cc:1196] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/0fd84ff103e14219aaae02bca1b13abf
I20260926 04:05:55.177030 16698 ts_tablet_manager.cc:1939] T 0fd84ff103e14219aaae02bca1b13abf P 09c137d28c984a6e8a6f0cec467b505b: Deleting consensus metadata
I20260926 04:05:55.177096 16183 catalog_manager.cc:4841] TS 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): tablet 0fd84ff103e14219aaae02bca1b13abf (table test-table [id=2aa994dfdd234781a1d2b7fca02f8167]) successfully deleted
I20260926 04:05:55.177441 16183 catalog_manager.cc:4841] TS 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): tablet 0fd84ff103e14219aaae02bca1b13abf (table test-table [id=2aa994dfdd234781a1d2b7fca02f8167]) successfully deleted
W20260926 04:05:55.178712 16183 connection.cc:535] client connection to 127.0.200.129:44489 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 04:05:55.178813 16183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 04:05:55.178966 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16394
I20260926 04:05:55.179556 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:05:55.179136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44489 (local address 127.0.0.1:57576)
0926 04:05:55.179482 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 04:05:55.179517 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327}
W20260926 04:05:55.180104 16183 catalog_manager.cc:4584] TS b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fd84ff103e14219aaae02bca1b13abf: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44489: connect: Connection refused (error 111)
I20260926 04:05:55.183631 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16528
I20260926 04:05:55.188089 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16166
I20260926 04:06:15.193538 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--webserver_interface=127.0.200.190
--webserver_port=46593
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:15.276441 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:15.276676 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:15.276714 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:15.277845 16700 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 04:06:15.277920 16700 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 04:06:15.277945 16700 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 04:06:15.277976 16700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:15.278002 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:15.278031 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:15.278061 16700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:15.278090 16700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:15.279779 16700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=46593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:15.280056 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:15.280287 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:15.289502 16709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.289515 16707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:15.289562 16700 server_base.cc:1034] running on GCE node
W20260926 04:06:15.289610 16706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:15.290038 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:15.290367 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:15.291515 16700 hybrid_clock.cc:648] HybridClock initialized: now 1790395575291507 us; error 34 us; skew 500 ppm
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:15.300216 16700 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:06:15.300781 16700 webserver.cc:460] Webserver started at http://127.0.200.190:46593/ using document root <none> and password file <none>
I20260926 04:06:15.301043 16700 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:15.301131 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:15.302680 16700 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:06:15.303375 16716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.303591 16700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.303701 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "54d03d80711a4a93a856f8ab0b1e7c87"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "8b60722ad9932cc55c6010893d97ebd9"
server_key_iv: "df91c7a023e094d90e76c752f206312d"
server_key_version: "encryptionkey@0"
I20260926 04:06:15.304139 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:15.323683 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:15.326864 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:15.327036 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:15.333746 16700 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:39941
I20260926 04:06:15.333957 16768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:39941 every 8 connection(s)
I20260926 04:06:15.334596 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:15.336745 16769 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:06:15.337464 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap starting.
I20260926 04:06:15.340436 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16700
I20260926 04:06:15.340791 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:44489
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=39987
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:06:15.350975 16769 log.cc:824] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:15.353814 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:06:15.354179 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap complete.
I20260926 04:06:15.358271 16769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:15.358705 16769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Initialized, Role: FOLLOWER
I20260926 04:06:15.358886 16769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:15.358981 16769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:15.359041 16769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:15.359084 16769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 1 FOLLOWER]: Advancing to term 2
I20260926 04:06:15.360661 16769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:15.360785 16769 leader_election.cc:302] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d03d80711a4a93a856f8ab0b1e7c87; no voters:
I20260926 04:06:15.360992 16769 leader_election.cc:288] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 04:06:15.361123 16773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 04:06:15.361310 16773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 LEADER]: Becoming Leader. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Running, Role: LEADER
I20260926 04:06:15.361357 16769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:15.361500 16773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:15.361853 16773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d03d80711a4a93a856f8ab0b1e7c87. Latest consensus state: current_term: 2 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:06:15.362146 16773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:15.362043 16774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:06:15.362602 16774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:15.362751 16782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:15.363557 16782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:15.363689 16782 catalog_manager.cc:1259] Loaded cluster ID: e9f9404d50784b87b11e2273d5712a3b
I20260926 04:06:15.363750 16782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:06:15.364576 16782 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:15.364775 16782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Loaded TSK: 0
I20260926 04:06:15.387866 16782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Generated new TSK 1
I20260926 04:06:15.388013 16782 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 04:06:15.457455 16771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:15.457757 16771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:15.457803 16771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:15.459555 16771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:15.459622 16771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:15.459666 16771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:15.461858 16771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=39987
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:15.462126 16771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:15.462374 16771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:15.467803 16797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.467912 16796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.467803 16799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:15.468590 16771 server_base.cc:1034] running on GCE node
I20260926 04:06:15.468833 16771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:15.469105 16771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:15.470288 16771 hybrid_clock.cc:648] HybridClock initialized: now 1790395575470277 us; error 32 us; skew 500 ppm
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:15.472930 16771 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:06:15.473423 16771 webserver.cc:460] Webserver started at http://127.0.200.129:39987/ using document root <none> and password file <none>
I20260926 04:06:15.473694 16771 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:15.473776 16771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:15.475281 16771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:15.476014 16806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.476212 16771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.476302 16771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b3eff65f5e3343479049af6691b2dcb6"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "6855ae92c2a904c17f2070ea7f26c84e"
server_key_iv: "8e9b2854c07a243bcf06a7bbab90e489"
server_key_version: "encryptionkey@0"
I20260926 04:06:15.476675 16771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:15.487318 16771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:15.489743 16771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:15.489936 16771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:15.490195 16771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:15.490716 16813 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 04:06:15.492715 16771 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 04:06:15.492792 16771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 04:06:15.492863 16771 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 04:06:15.493528 16771 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 04:06:15.493587 16771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 04:06:15.493641 16813 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Bootstrap starting.
I20260926 04:06:15.501006 16771 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:44489
I20260926 04:06:15.501102 16920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:44489 every 8 connection(s)
I20260926 04:06:15.501497 16771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:15.504243 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16771
I20260926 04:06:15.504568 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:34635
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=41813
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:06:15.504967 16813 log.cc:824] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:15.507835 16813 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:06:15.508180 16813 tablet_bootstrap.cc:492] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Bootstrap complete.
I20260926 04:06:15.509382 16813 ts_tablet_manager.cc:1397] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260926 04:06:15.510869 16813 raft_consensus.cc:348] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.511211 16813 raft_consensus.cc:729] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eff65f5e3343479049af6691b2dcb6, State: Initialized, Role: FOLLOWER
I20260926 04:06:15.511373 16813 consensus_queue.cc:260] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.511646 16813 ts_tablet_manager.cc:1428] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:15.521065 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.502340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:55201)
0926 04:06:15.506332 (+ 3992us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.506386 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.508333 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.509055 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.509060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.509559 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:15.509877 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.509885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.512006 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.512007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.513155 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.513158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.514122 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.514124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.514260 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.514988 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:15.514998 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:15.516837 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:15.519165 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.519172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.519174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.519476 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.519772 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.519773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.519773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.519901 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.520391 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:15.520398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:15.520562 (+ 164us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.520675 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.520846 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3941,"thread_start_us":113,"threads_started":1}
I20260926 04:06:15.521773 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.502420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:55201 (local address 127.0.200.190:39941)
0926 04:06:15.506677 (+ 4257us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.506686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.506736 (+ 50us) server_negotiation.cc:407] Connection header received
0926 04:06:15.508648 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.508655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.508739 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.508884 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:15.510030 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.511878 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.513266 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.514003 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.514126 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.516992 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:15.516996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:15.516997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:15.517008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:15.519067 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.519575 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.519578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.519579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.519693 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.519980 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.519981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.519982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.520311 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:15.520403 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.520763 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.520902 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4199,"thread_start_us":76,"threads_started":1}
I20260926 04:06:15.522647 16921 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:06:15.522750 16921 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:15.522945 16921 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:06:15.523769 16726 ts_manager.cc:194] Registered new tserver with Master: b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489)
I20260926 04:06:15.524967 16726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:55201
I20260926 04:06:15.525964 16921 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:06:15.530974 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.524695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.0.200.129:44489)
0926 04:06:15.524863 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.524868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.526741 (+ 1873us) server_negotiation.cc:407] Connection header received
0926 04:06:15.526887 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.526889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.526905 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.526970 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:15.527245 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.528282 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.529770 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.530458 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.530505 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.530579 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.530837 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.530879 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260926 04:06:15.531399 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.524415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44489 (local address 127.0.0.1:54718)
0926 04:06:15.526639 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.526678 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.526780 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.527065 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.527068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.527071 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:15.527161 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.527162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.528407 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.528408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.529664 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.529667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.530638 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.530639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.530731 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.530743 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.531252 (+ 509us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.531319 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2194,"thread_start_us":113,"threads_started":1}
I20260926 04:06:15.531952 16855 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd84ff103e14219aaae02bca1b13abf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:05:55 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:54718
I20260926 04:06:15.532572 16930 tablet_replica.cc:330] stopping tablet replica
I20260926 04:06:15.532686 16930 raft_consensus.cc:2229] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:15.532752 16930 raft_consensus.cc:2258] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:15.533104 16930 ts_tablet_manager.cc:1905] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:15.535683 16930 ts_tablet_manager.cc:1918] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:15.535765 16930 log.cc:1196] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal/wals/0fd84ff103e14219aaae02bca1b13abf
I20260926 04:06:15.535983 16930 ts_tablet_manager.cc:1939] T 0fd84ff103e14219aaae02bca1b13abf P b3eff65f5e3343479049af6691b2dcb6: Deleting consensus metadata
I20260926 04:06:15.536549 16718 catalog_manager.cc:4841] TS b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489): tablet 0fd84ff103e14219aaae02bca1b13abf (table test-table [id=2aa994dfdd234781a1d2b7fca02f8167]) successfully deleted
W20260926 04:06:15.596719 16926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:15.596951 16926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:15.596977 16926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:15.598106 16926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:15.598146 16926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:15.598173 16926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:15.599606 16926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:34635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=41813
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:15.599794 16926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:15.599979 16926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:15.605116 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.605140 16936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.605180 16935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:15.605319 16926 server_base.cc:1034] running on GCE node
I20260926 04:06:15.605621 16926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:15.605842 16926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:15.607033 16926 hybrid_clock.cc:648] HybridClock initialized: now 1790395575607032 us; error 53 us; skew 500 ppm
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:15.609859 16926 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:06:15.610428 16926 webserver.cc:460] Webserver started at http://127.0.200.130:41813/ using document root <none> and password file <none>
I20260926 04:06:15.610688 16926 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:15.610749 16926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:15.612140 16926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.612854 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.613089 16926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.613181 16926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "d1456123a667b886f42f28a9dac16daf"
server_key_iv: "e555190ead378b51eaf0c308c2a562c2"
server_key_version: "encryptionkey@0"
I20260926 04:06:15.613545 16926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:15.632176 16926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:15.634382 16926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:15.634514 16926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:15.634806 16926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:15.635071 16926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:15.635102 16926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.635125 16926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:15.635138 16926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.641355 16926 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:34635
I20260926 04:06:15.641453 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:34635 every 8 connection(s)
I20260926 04:06:15.641768 16926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:15.651104 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 16926
I20260926 04:06:15.651423 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:35555
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=41393
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:06:15.651976 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.642602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:55839 (local address 127.0.200.190:39941)
0926 04:06:15.642654 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.642656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.642927 (+ 271us) server_negotiation.cc:407] Connection header received
0926 04:06:15.643488 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.643490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.643509 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.643565 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:15.644606 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.645125 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.646198 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.646427 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.646532 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.648918 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:15.648920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:15.648921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:15.648928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:15.650389 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.650917 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.650918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.650918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.650988 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.651272 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.651274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.651274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.651405 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:15.651487 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.651772 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.651880 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:06:15.652202 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.642632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:55839)
0926 04:06:15.642795 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.642831 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.643358 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.643684 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.643688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.644293 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:15.644494 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.644499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.645226 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.645227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.646094 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.646096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.646584 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.646585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.646719 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.647305 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:15.647313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:15.648816 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:15.650490 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.650497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.650499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.650838 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.651075 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.651075 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.651075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.651178 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.651477 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:15.651484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:15.651628 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.651736 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.651902 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260926 04:06:15.652745 17059 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:06:15.652868 17059 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:15.653075 17059 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:06:15.653442 16726 ts_manager.cc:194] Registered new tserver with Master: 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635)
I20260926 04:06:15.653823 16726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:55839
W20260926 04:06:15.733318 17062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:15.733572 17062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:15.733608 17062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:15.734921 17062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:15.734995 17062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:15.735040 17062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:15.736492 17062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:35555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=41393
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:15.736768 17062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:15.737020 17062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:06:15.742296 17062 server_base.cc:1034] running on GCE node
W20260926 04:06:15.742251 17067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.742286 17068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:15.742259 17070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:15.742906 17062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:15.743158 17062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:15.744360 17062 hybrid_clock.cc:648] HybridClock initialized: now 1790395575744345 us; error 30 us; skew 500 ppm
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:15.747182 17062 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:06:15.747628 17062 webserver.cc:460] Webserver started at http://127.0.200.131:41393/ using document root <none> and password file <none>
I20260926 04:06:15.747828 17062 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:15.747881 17062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:15.749279 17062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.749975 17077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.750193 17062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.750286 17062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "09c137d28c984a6e8a6f0cec467b505b"
format_stamp: "Formatted at 2026-09-26 04:05:55 on dist-test-slave-j8c1"
server_key: "9514654093fdc37d50219ef665d0ba22"
server_key_iv: "1f98a7ce7823e19a84fb2fd8ef34b360"
server_key_version: "encryptionkey@0"
I20260926 04:06:15.750748 17062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:15.764073 17062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:15.766476 17062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:15.766665 17062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:15.766886 17062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:15.767280 17062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:15.767318 17062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.767367 17062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:15.767393 17062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:15.773154 17062 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:35555
I20260926 04:06:15.773254 17190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:35555 every 8 connection(s)
I20260926 04:06:15.773584 17062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:15.777076 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17062
Sep 26 04:06:15 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:15.783725 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.774360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:54197 (local address 127.0.200.190:39941)
0926 04:06:15.774407 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.774409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.774715 (+ 306us) server_negotiation.cc:407] Connection header received
0926 04:06:15.775260 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.775262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.775276 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.775331 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:15.776318 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.776841 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.778082 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.778287 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.778368 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.780912 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:15.780914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:15.780914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:15.780922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:15.782237 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.782759 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.782760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.782761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.782824 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.783078 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.783078 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.783079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.783193 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:15.783272 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.783546 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.783627 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:06:15.783957 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.774344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:54197)
0926 04:06:15.774535 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.774618 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.775143 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.775417 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.775421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.776010 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:15.776215 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.776220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.776940 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.776941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.777989 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.777992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.778376 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.778377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.778509 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.779159 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:15.779167 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:15.780815 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:15.782360 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.782367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.782370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.782676 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.782890 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:15.782891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:15.782891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:15.783005 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:15.783261 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:15.783267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:15.783413 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.783525 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.783692 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260926 04:06:15.784407 17191 heartbeater.cc:347] Connected to a master server at 127.0.200.190:39941
I20260926 04:06:15.784543 17191 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:15.784722 17191 heartbeater.cc:511] Master 127.0.200.190:39941 requested a full tablet report, sending...
I20260926 04:06:15.784986 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.777547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.0.200.190:39941)
0926 04:06:15.777665 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.777667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.777772 (+ 105us) server_negotiation.cc:407] Connection header received
0926 04:06:15.777850 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.777851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.777864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.777946 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:15.778524 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.779265 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.780240 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.780485 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.780540 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.781974 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:15.781975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:15.781976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:15.781982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:15.783542 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.783983 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.783984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.783984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.784185 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:15.784428 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:15.784429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:15.784430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:15.784577 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:15.784763 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.784800 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.784882 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
I20260926 04:06:15.785110 16726 ts_manager.cc:194] Registered new tserver with Master: 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
I20260926 04:06:15.785480 16726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:54197
I20260926 04:06:15.786104 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:15.789160 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.786491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.0.200.190:39941)
0926 04:06:15.786573 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.786575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.786645 (+ 70us) server_negotiation.cc:407] Connection header received
0926 04:06:15.786698 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.786698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.786728 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.786788 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:15.786877 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.787321 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.788174 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.788388 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.788455 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.788671 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:15.788767 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:15.788844 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.789015 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.789038 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 04:06:15.789523 16726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:15.791347 16726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:15.793828 16855 tablet_service.cc:1467] Processing CreateTablet for tablet a3f8b119819242d48433fce0ffc2da0e (DEFAULT_TABLE table=test-table [id=dac077e2ee854dcc87ab5bdda7e0ddaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:15.794052 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3f8b119819242d48433fce0ffc2da0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:15.795704 16813 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Bootstrap starting.
I20260926 04:06:15.796051 16813 tablet_bootstrap.cc:654] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:15.796641 16813 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: No bootstrap required, opened a new log
I20260926 04:06:15.796701 16813 ts_tablet_manager.cc:1397] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:15.796821 16813 raft_consensus.cc:348] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.796896 16813 raft_consensus.cc:374] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:15.796939 16813 raft_consensus.cc:729] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eff65f5e3343479049af6691b2dcb6, State: Initialized, Role: FOLLOWER
I20260926 04:06:15.797003 16813 consensus_queue.cc:260] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.797091 16813 ts_tablet_manager.cc:1428] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260926 04:06:15.797205 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.793311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34635 (local address 127.0.0.1:35980)
0926 04:06:15.793364 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.793376 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.793423 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.793899 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.793901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.793903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:15.793949 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.793951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.795021 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.795022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.796034 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.796037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.796967 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.796968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.797033 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.797037 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.797081 (+ 44us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.797129 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 04:06:15.797628 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.793531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.0.200.130:34635)
0926 04:06:15.793717 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.793722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.793734 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:06:15.793780 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.793782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.793800 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.793883 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:15.794012 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.794933 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.796127 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.796865 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.797393 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.797469 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.797496 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.797543 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260926 04:06:15.798357 16993 tablet_service.cc:1467] Processing CreateTablet for tablet a3f8b119819242d48433fce0ffc2da0e (DEFAULT_TABLE table=test-table [id=dac077e2ee854dcc87ab5bdda7e0ddaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:15.798528 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.793408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35555 (local address 127.0.0.1:37982)
0926 04:06:15.793739 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.793751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.793799 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.795215 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.795216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.795218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:15.795268 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.795270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.796334 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.796335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.797422 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.797424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.798272 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.798273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.798341 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.798348 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.798419 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.798462 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":110,"threads_started":1}
I20260926 04:06:15.798666 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.793514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.0.200.131:35555)
0926 04:06:15.795067 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.795070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.795078 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:06:15.795112 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.795113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.795126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.795217 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:15.795329 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.796237 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.797525 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.798185 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.798245 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.798277 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.798446 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.798523 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":132,"threads_started":1}
I20260926 04:06:15.798805 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3f8b119819242d48433fce0ffc2da0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:15.799276 17125 tablet_service.cc:1467] Processing CreateTablet for tablet a3f8b119819242d48433fce0ffc2da0e (DEFAULT_TABLE table=test-table [id=dac077e2ee854dcc87ab5bdda7e0ddaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:15.799707 17125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3f8b119819242d48433fce0ffc2da0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:15.801782 17201 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Bootstrap starting.
I20260926 04:06:15.802491 17202 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Bootstrap starting.
I20260926 04:06:15.802747 17201 tablet_bootstrap.cc:654] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:15.803196 17202 tablet_bootstrap.cc:654] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:15.803383 17201 log.cc:824] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:15.803737 17202 log.cc:824] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:15.804250 17201 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: No bootstrap required, opened a new log
I20260926 04:06:15.804327 17201 ts_tablet_manager.cc:1397] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 04:06:15.804451 17202 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: No bootstrap required, opened a new log
I20260926 04:06:15.804514 17202 ts_tablet_manager.cc:1397] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:06:15.806375 17201 raft_consensus.cc:348] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.806401 17202 raft_consensus.cc:348] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.806625 17202 raft_consensus.cc:374] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:15.806625 17201 raft_consensus.cc:374] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:15.806665 17201 raft_consensus.cc:729] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a74c0ea83e4e5ca36d2ebbb3bb0acd, State: Initialized, Role: FOLLOWER
I20260926 04:06:15.806665 17202 raft_consensus.cc:729] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c137d28c984a6e8a6f0cec467b505b, State: Initialized, Role: FOLLOWER
I20260926 04:06:15.806797 17202 consensus_queue.cc:260] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.806813 17201 consensus_queue.cc:260] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.807107 17191 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:06:15.807099 17201 ts_tablet_manager.cc:1428] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:15.807103 17059 heartbeater.cc:503] Master 127.0.200.190:39941 was elected leader, sending a full tablet report...
I20260926 04:06:15.807178 17202 ts_tablet_manager.cc:1428] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:15.860431 17206 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:15.860562 17206 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.860988 17206 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489)
I20260926 04:06:15.864434 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.861422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33659 (local address 127.0.200.129:44489)
0926 04:06:15.861492 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.861494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.861634 (+ 140us) server_negotiation.cc:407] Connection header received
0926 04:06:15.861695 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.861696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.861724 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.861785 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:15.861962 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.862604 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.863691 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.864073 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.864130 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.864164 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.864312 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.864366 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 04:06:15.864444 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.861367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44489 (local address 127.0.200.131:33659)
0926 04:06:15.861556 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.861567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.861630 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.861814 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.861815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.861817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:15.861893 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.861895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.862689 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.862690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.863598 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.863600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.864158 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.864159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.864231 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.864240 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.864298 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.864346 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260926 04:06:15.864573 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.861353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:38951 (local address 127.0.200.130:34635)
0926 04:06:15.861415 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.861418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.861429 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:15.861468 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.861469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.861498 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.861633 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:15.861835 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.862491 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.863699 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.864063 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.864171 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.864179 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.864445 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.864492 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:06:15.864809 16875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3f8b119819242d48433fce0ffc2da0e" candidate_uuid: "09c137d28c984a6e8a6f0cec467b505b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3eff65f5e3343479049af6691b2dcb6" is_pre_election: true
I20260926 04:06:15.864895 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.861220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34635 (local address 127.0.200.131:38951)
0926 04:06:15.861280 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.861292 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:15.861340 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:15.861674 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:15.861675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:15.861677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:15.861754 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.861757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.862607 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.862608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.863598 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:15.863601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.864187 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.864188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:15.864313 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.864318 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:06:15.864369 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 04:06:15.864407 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":60}
I20260926 04:06:15.864987 16875 raft_consensus.cc:2454] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c137d28c984a6e8a6f0cec467b505b in term 0.
I20260926 04:06:15.865250 17079 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c137d28c984a6e8a6f0cec467b505b, b3eff65f5e3343479049af6691b2dcb6; no voters:
I20260926 04:06:15.865252 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3f8b119819242d48433fce0ffc2da0e" candidate_uuid: "09c137d28c984a6e8a6f0cec467b505b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" is_pre_election: true
I20260926 04:06:15.865382 17013 raft_consensus.cc:2454] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09c137d28c984a6e8a6f0cec467b505b in term 0.
I20260926 04:06:15.865406 17206 raft_consensus.cc:2781] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:15.865460 17206 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:15.865495 17206 raft_consensus.cc:3037] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:15.866194 17206 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.866333 17206 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [CANDIDATE]: Term 1 election: Requested vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489)
I20260926 04:06:15.866473 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3f8b119819242d48433fce0ffc2da0e" candidate_uuid: "09c137d28c984a6e8a6f0cec467b505b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
I20260926 04:06:15.866521 16875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3f8b119819242d48433fce0ffc2da0e" candidate_uuid: "09c137d28c984a6e8a6f0cec467b505b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3eff65f5e3343479049af6691b2dcb6"
I20260926 04:06:15.866597 17013 raft_consensus.cc:3037] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:15.866642 16875 raft_consensus.cc:3037] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:15.867341 17013 raft_consensus.cc:2454] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c137d28c984a6e8a6f0cec467b505b in term 1.
I20260926 04:06:15.867525 17079 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd; no voters:
I20260926 04:06:15.867625 17206 raft_consensus.cc:2781] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:15.867802 17206 raft_consensus.cc:686] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 1 LEADER]: Becoming Leader. State: Replica: 09c137d28c984a6e8a6f0cec467b505b, State: Running, Role: LEADER
I20260926 04:06:15.867902 16875 raft_consensus.cc:2454] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09c137d28c984a6e8a6f0cec467b505b in term 1.
I20260926 04:06:15.867916 17206 consensus_queue.cc:237] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:15.868876 16733 catalog_manager.cc:5495] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b reported cstate change: term changed from 0 to 1, leader changed from <none> to 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131). New cstate: current_term: 1 leader_uuid: "09c137d28c984a6e8a6f0cec467b505b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } health_report { overall_health: UNKNOWN } } }
I20260926 04:06:15.891757 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.889273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.0.200.131:35555)
0926 04:06:15.889344 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:15.889345 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:15.889352 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:06:15.889385 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:15.889386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:15.889402 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:15.889477 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:15.889670 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.890144 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.890944 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:15.891149 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:15.891226 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:15.891402 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:15.891498 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:15.891590 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:15.891617 (+ 27us) server_negotiation.cc:299] Negotiation successful
0926 04:06:15.891638 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 04:06:15.892342 17060 tablet.cc:2369] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:15.893661 17013 raft_consensus.cc:1261] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Refusing update from remote peer 09c137d28c984a6e8a6f0cec467b505b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:15.893662 16875 raft_consensus.cc:1261] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Refusing update from remote peer 09c137d28c984a6e8a6f0cec467b505b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:15.893997 17206 consensus_queue.cc:1035] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [LEADER]: Connected to new peer: Peer: permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:15.894196 17208 consensus_queue.cc:1035] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:15.896904 17212 mvcc.cc:205] Tried to move back new op lower bound from 7333460278857617408 to 7333460278755635200. Current Snapshot: MvccSnapshot[applied={T|T < 7333460278857617408}]
I20260926 04:06:15.896914 17213 mvcc.cc:205] Tried to move back new op lower bound from 7333460278857617408 to 7333460278755635200. Current Snapshot: MvccSnapshot[applied={T|T < 7333460278857617408}]
I20260926 04:06:15.898896 16733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:15.899025 16733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:15.899878 16733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Sending DeleteTablet for 3 replicas of tablet a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:15.900247 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16771
I20260926 04:06:15.900243 17125 tablet_service.cc:1514] Processing DeleteTablet for tablet a3f8b119819242d48433fce0ffc2da0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:15 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:37982
I20260926 04:06:15.900350 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet a3f8b119819242d48433fce0ffc2da0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:15 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:35980
I20260926 04:06:15.900612 17222 tablet_replica.cc:330] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: stopping tablet replica
I20260926 04:06:15.900720 17222 raft_consensus.cc:2229] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:15.900899 17222 raft_consensus.cc:2258] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:15.902024 17221 tablet_replica.cc:330] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: stopping tablet replica
I20260926 04:06:15.902068 17222 ts_tablet_manager.cc:1905] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:15.902125 17221 raft_consensus.cc:2229] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:15.902249 17221 raft_consensus.cc:2258] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:15.902729 17221 ts_tablet_manager.cc:1905] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:15.904160 17222 ts_tablet_manager.cc:1918] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:15.904240 17222 log.cc:1196] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:15.904533 17222 ts_tablet_manager.cc:1939] T a3f8b119819242d48433fce0ffc2da0e P 18a74c0ea83e4e5ca36d2ebbb3bb0acd: Deleting consensus metadata
I20260926 04:06:15.904721 17221 ts_tablet_manager.cc:1918] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:15.904784 17221 log.cc:1196] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:15.905043 17221 ts_tablet_manager.cc:1939] T a3f8b119819242d48433fce0ffc2da0e P 09c137d28c984a6e8a6f0cec467b505b: Deleting consensus metadata
I20260926 04:06:15.905700 16718 catalog_manager.cc:4841] TS 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): tablet a3f8b119819242d48433fce0ffc2da0e (table test-table [id=dac077e2ee854dcc87ab5bdda7e0ddaa]) successfully deleted
I20260926 04:06:15.905799 16718 catalog_manager.cc:4841] TS 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): tablet a3f8b119819242d48433fce0ffc2da0e (table test-table [id=dac077e2ee854dcc87ab5bdda7e0ddaa]) successfully deleted
W20260926 04:06:15.907414 16718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:44489 (error 108)
I20260926 04:06:15.907948 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16926
I20260926 04:06:15.908100 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:15.907950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:44489 (local address 127.0.0.1:54726)
0926 04:06:15.908015 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:15.908058 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 04:06:15.908702 16718 catalog_manager.cc:4584] TS b3eff65f5e3343479049af6691b2dcb6 (127.0.200.129:44489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3f8b119819242d48433fce0ffc2da0e: Network error: Client connection negotiation failed: client connection to 127.0.200.129:44489: connect: Connection refused (error 111)
I20260926 04:06:15.913682 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17062
I20260926 04:06:15.919065 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 16700
I20260926 04:06:35.924872 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--webserver_interface=127.0.200.190
--webserver_port=46593
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:36.011936 17224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:36.012154 17224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:36.012178 17224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:36.013374 17224 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 04:06:36.013422 17224 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 04:06:36.013434 17224 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 04:06:36.013446 17224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:36.013459 17224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:36.013473 17224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:36.013527 17224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:36.013545 17224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:36.015149 17224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=46593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:36.015407 17224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:36.015668 17224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:36.020920 17230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.020995 17232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.021013 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:36.021562 17224 server_base.cc:1034] running on GCE node
I20260926 04:06:36.021811 17224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:36.022149 17224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:36.023337 17224 hybrid_clock.cc:648] HybridClock initialized: now 1790395596023320 us; error 47 us; skew 500 ppm
Sep 26 04:06:36 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:36.026121 17224 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:06:36.026628 17224 webserver.cc:460] Webserver started at http://127.0.200.190:46593/ using document root <none> and password file <none>
I20260926 04:06:36.026873 17224 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:36.026921 17224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:36.028400 17224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:36.029209 17239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.029429 17224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.029521 17224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "54d03d80711a4a93a856f8ab0b1e7c87"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "8b60722ad9932cc55c6010893d97ebd9"
server_key_iv: "df91c7a023e094d90e76c752f206312d"
server_key_version: "encryptionkey@0"
I20260926 04:06:36.029883 17224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:36.058597 17224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:36.061040 17224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:36.061167 17224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:36.065147 17224 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:39941
I20260926 04:06:36.065209 17291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:39941 every 8 connection(s)
I20260926 04:06:36.065598 17224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:36.067322 17292 sys_catalog.cc:263] Verifying existing consensus state
I20260926 04:06:36.068084 17292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap starting.
I20260926 04:06:36.071926 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17224
I20260926 04:06:36.072321 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:44489
--local_ip_for_outbound_sockets=127.0.200.129
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=39987
--webserver_interface=127.0.200.129
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:06:36.077426 17292 log.cc:824] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:36.080360 17292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:06:36.080698 17292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Bootstrap complete.
I20260926 04:06:36.083168 17292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:36.083449 17292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Initialized, Role: FOLLOWER
I20260926 04:06:36.083657 17292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:36.083742 17292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:36.083796 17292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:36.083849 17292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 2 FOLLOWER]: Advancing to term 3
I20260926 04:06:36.084944 17292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:36.085080 17292 leader_election.cc:302] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d03d80711a4a93a856f8ab0b1e7c87; no voters:
I20260926 04:06:36.085306 17292 leader_election.cc:288] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 04:06:36.085351 17296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 04:06:36.085477 17296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [term 3 LEADER]: Becoming Leader. State: Replica: 54d03d80711a4a93a856f8ab0b1e7c87, State: Running, Role: LEADER
I20260926 04:06:36.085572 17296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } }
I20260926 04:06:36.085608 17292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:36.085889 17297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:06:36.085984 17297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:36.086228 17298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d03d80711a4a93a856f8ab0b1e7c87. Latest consensus state: current_term: 3 leader_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d03d80711a4a93a856f8ab0b1e7c87" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39941 } } }
I20260926 04:06:36.086331 17298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:36.086375 17301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:36.087317 17301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:36.087425 17301 catalog_manager.cc:1259] Loaded cluster ID: e9f9404d50784b87b11e2273d5712a3b
I20260926 04:06:36.087450 17301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:06:36.088694 17301 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:36.088888 17301 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Loaded TSK: 0
I20260926 04:06:36.088949 17301 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54d03d80711a4a93a856f8ab0b1e7c87: Loaded TSK: 1
F20260926 04:06:36.089190 17301 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6ac04a19ad google::LogMessage::Fail() at ??:0
@ 0x7f6ac04a488a google::LogMessage::SendToLog() at ??:0
@ 0x7f6ac04a14de google::LogMessage::Flush() at ??:0
@ 0x7f6ac04a2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6ac2c0e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6ac2c0bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6ac83ff1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6ac83ff48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6ac83ff594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6ac83c5c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6ac83cfad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6ac236f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6ac2364954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6ac31c86db start_thread at ??:0
@ 0x7f6abef9f61f clone at ??:0
W20260926 04:06:36.186379 17294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:36.186715 17294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:36.186760 17294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:36.188591 17294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:36.188661 17294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:36.188706 17294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:36.191018 17294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=39987
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:36.191313 17294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:36.191601 17294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:36.197417 17336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.197391 17333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.197397 17334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:36.197592 17294 server_base.cc:1034] running on GCE node
I20260926 04:06:36.197983 17294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:36.198278 17294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:36.199478 17294 hybrid_clock.cc:648] HybridClock initialized: now 1790395596199427 us; error 76 us; skew 500 ppm
Sep 26 04:06:36 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:36.202316 17294 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:06:36.202925 17294 webserver.cc:460] Webserver started at http://127.0.200.129:39987/ using document root <none> and password file <none>
I20260926 04:06:36.203210 17294 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:36.203261 17294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:36.204751 17294 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:36.205530 17343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.205768 17294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.205924 17294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "b3eff65f5e3343479049af6691b2dcb6"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "6855ae92c2a904c17f2070ea7f26c84e"
server_key_iv: "8e9b2854c07a243bcf06a7bbab90e489"
server_key_version: "encryptionkey@0"
I20260926 04:06:36.206328 17294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:36.220676 17294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:36.223414 17294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:36.223604 17294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:36.223886 17294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:36.224383 17350 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 04:06:36.226424 17294 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 04:06:36.226508 17294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260926 04:06:36.226609 17294 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 04:06:36.227349 17294 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 04:06:36.227415 17294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260926 04:06:36.227487 17350 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Bootstrap starting.
I20260926 04:06:36.234524 17294 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:44489
I20260926 04:06:36.234745 17457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:44489 every 8 connection(s)
I20260926 04:06:36.235051 17294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:36.238454 17350 log.cc:824] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:36.239117 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17294
I20260926 04:06:36.239410 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:34635
--local_ip_for_outbound_sockets=127.0.200.130
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=41813
--webserver_interface=127.0.200.130
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:06:36.241413 17350 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 04:06:36.241727 17350 tablet_bootstrap.cc:492] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Bootstrap complete.
I20260926 04:06:36.242972 17350 ts_tablet_manager.cc:1397] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260926 04:06:36.244479 17350 raft_consensus.cc:348] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:36.244817 17350 raft_consensus.cc:729] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eff65f5e3343479049af6691b2dcb6, State: Initialized, Role: FOLLOWER
I20260926 04:06:36.244992 17350 consensus_queue.cc:260] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:36.245280 17350 ts_tablet_manager.cc:1428] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260926 04:06:36.314759 17460 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:06:36.235962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:40527)
0926 04:06:36.236156 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.236204 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:36.236717 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:36.314462 (+ 77745us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
W20260926 04:06:36.315084 17458 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:39941 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:39941: Client connection negotiation failed: client connection to 127.0.200.190:39941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 04:06:36.315503 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.315349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:35075)
0926 04:06:36.315408 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.315467 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 04:06:36.315907 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.315787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:60463)
0926 04:06:36.315849 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.315883 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 04:06:36.316046 17458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 04:06:36.334053 17462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:36.334314 17462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:36.334344 17462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:36.335677 17462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:36.335724 17462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:36.335788 17462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:36.337448 17462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:34635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=41813
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:36.337706 17462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:36.337992 17462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:36.343115 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.343115 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.343218 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:36.343263 17462 server_base.cc:1034] running on GCE node
I20260926 04:06:36.343528 17462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:36.343775 17462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:36.344933 17462 hybrid_clock.cc:648] HybridClock initialized: now 1790395596344920 us; error 29 us; skew 500 ppm
Sep 26 04:06:36 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:36.347587 17462 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:06:36.348075 17462 webserver.cc:460] Webserver started at http://127.0.200.130:41813/ using document root <none> and password file <none>
I20260926 04:06:36.348306 17462 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:36.348377 17462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:36.349799 17462 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.350623 17478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.350817 17462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.350960 17462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd"
format_stamp: "Formatted at 2026-09-26 04:05:54 on dist-test-slave-j8c1"
server_key: "d1456123a667b886f42f28a9dac16daf"
server_key_iv: "e555190ead378b51eaf0c308c2a562c2"
server_key_version: "encryptionkey@0"
I20260926 04:06:36.351336 17462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:36.363138 17462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:36.365046 17462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:36.365218 17462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:36.365463 17462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:36.365774 17462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:36.365828 17462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.365875 17462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:36.365902 17462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.371992 17462 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:34635
I20260926 04:06:36.372030 17591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:34635 every 8 connection(s)
I20260926 04:06:36.372394 17462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:36.373550 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.373130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:49929)
0926 04:06:36.373324 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.373401 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
W20260926 04:06:36.373976 17592 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:39941 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:39941: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
I20260926 04:06:36.374296 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.374178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:36245)
0926 04:06:36.374240 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.374270 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 04:06:36.374719 17594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.374625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:58847)
0926 04:06:36.374678 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.374703 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 04:06:36.374879 17592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 04:06:36.377342 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17462
I20260926 04:06:36.377630 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:35555
--local_ip_for_outbound_sockets=127.0.200.131
--tserver_master_addrs=127.0.200.190:39941
--webserver_port=41393
--webserver_interface=127.0.200.131
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:36.458179 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:36.458380 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:36.458403 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:36.459568 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:36.459612 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:36.459647 17595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:36.461133 17595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:35555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=41393
--tserver_master_addrs=127.0.200.190:39941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:36.461387 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:36.461627 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:06:36.466773 17595 server_base.cc:1034] running on GCE node
W20260926 04:06:36.466790 17601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.466791 17600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:36.466755 17603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:36.467224 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:36.467473 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:36.468657 17595 hybrid_clock.cc:648] HybridClock initialized: now 1790395596468637 us; error 37 us; skew 500 ppm
Sep 26 04:06:36 dist-test-slave-j8c1 krb5kdc[16153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:36.471160 17595 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:06:36.471587 17595 webserver.cc:460] Webserver started at http://127.0.200.131:41393/ using document root <none> and password file <none>
I20260926 04:06:36.471834 17595 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:36.471881 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:36.473265 17595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.473892 17610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.474081 17595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:36.474180 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "09c137d28c984a6e8a6f0cec467b505b"
format_stamp: "Formatted at 2026-09-26 04:05:55 on dist-test-slave-j8c1"
server_key: "9514654093fdc37d50219ef665d0ba22"
server_key_iv: "1f98a7ce7823e19a84fb2fd8ef34b360"
server_key_version: "encryptionkey@0"
I20260926 04:06:36.474534 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:36.483021 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:36.484894 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:36.485054 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:36.485273 17595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:36.485571 17595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:36.485631 17595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.485675 17595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:36.485703 17595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:36.491674 17595 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:35555
I20260926 04:06:36.491716 17723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:35555 every 8 connection(s)
I20260926 04:06:36.492126 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:36.492492 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17595
I20260926 04:06:36.493307 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.492868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:48775)
0926 04:06:36.493029 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.493105 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
W20260926 04:06:36.493678 17724 heartbeater.cc:644] Failed to heartbeat to 127.0.200.190:39941 (0 consecutive failures): Network error: Failed to ping master at 127.0.200.190:39941: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
I20260926 04:06:36.493954 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.493858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:36615)
0926 04:06:36.493908 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.493932 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260926 04:06:36.494225 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:36.494135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:50605)
0926 04:06:36.494183 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:36.494205 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260926 04:06:36.494331 17724 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 04:06:37.316781 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.316515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:57481)
0926 04:06:37.316665 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:37.316711 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260926 04:06:37.375625 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.375369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:39701)
0926 04:06:37.375509 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:37.375563 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260926 04:06:37.494949 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.494716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:35577)
0926 04:06:37.494856 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:37.494893 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":58,"threads_started":1}
I20260926 04:06:37.508785 17731 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:37.509001 17731 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:37.509387 17731 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
Sep 26 04:06:37 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.130@KRBTEST.COM
Sep 26 04:06:37 dist-test-slave-j8c1 krb5kdc[16153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.131@KRBTEST.COM
I20260926 04:06:37.520394 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.509874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:50209 (local address 127.0.200.131:35555)
0926 04:06:37.510018 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:37.510026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:37.510052 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:06:37.510595 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:37.510598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:37.510649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:37.510809 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:37.511537 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.512391 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513386 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.513786 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513919 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:37.516815 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:37.516819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:37.516819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:37.516828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:37.518865 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:37.519268 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:37.519271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:37.519271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:37.519374 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:37.519575 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:37.519575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:37.519576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:37.519872 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:37.519951 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:37.520136 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 04:06:37.520273 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260926 04:06:37.520387 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.509589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34635 (local address 127.0.200.129:53337)
0926 04:06:37.509663 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:37.509679 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:37.509753 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:37.510831 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:37.510834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:37.511222 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:37.511407 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:37.511413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.512483 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.512484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:37.513294 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:37.513296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.513893 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:37.514004 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:37.514721 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:37.514726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:37.516290 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:37.518538 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:37.518574 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:37.518577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:37.518906 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:37.519176 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:37.519177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:37.519177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:37.519298 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:37.519808 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:37.519814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:37.519955 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 04:06:37.520066 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 04:06:37.520268 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":56}
I20260926 04:06:37.520411 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.509768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:53337 (local address 127.0.200.130:34635)
0926 04:06:37.509933 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:37.509946 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:37.509981 (+ 35us) server_negotiation.cc:407] Connection header received
0926 04:06:37.510589 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:37.510592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:37.510649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:37.510797 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:37.511537 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.512376 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513385 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.513784 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513914 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:37.516397 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:37.516401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:37.516401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:37.516411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:37.518438 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:37.518992 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:37.518995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:37.518996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:37.519100 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:37.519374 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:37.519376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:37.519376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:37.519726 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:37.519813 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:37.520150 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 04:06:37.520294 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260926 04:06:37.520462 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:37.509718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:35555 (local address 127.0.200.129:50209)
0926 04:06:37.509837 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:37.509847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:37.509911 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:37.510832 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:37.510834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:37.511221 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:37.511407 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:37.511413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.512503 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.512504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:37.513295 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:37.513297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:37.513890 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:37.513891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:37.514004 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:37.514640 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:37.514647 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:37.516698 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:37.518955 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:37.518956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:37.518956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:37.519175 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:37.519440 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:37.519440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:37.519441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:37.519496 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:37.519934 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:37.519934 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:37.519971 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 04:06:37.520086 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 04:06:37.520330 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":84,"thread_start_us":61,"threads_started":1}
W20260926 04:06:37.521785 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
W20260926 04:06:37.521884 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:37.521917 17345 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3eff65f5e3343479049af6691b2dcb6; no voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd
I20260926 04:06:37.522037 17731 raft_consensus.cc:2726] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:06:38.317529 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:38.317287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:59547)
0926 04:06:38.317437 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:38.317481 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260926 04:06:38.376493 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:38.376240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:49971)
0926 04:06:38.376403 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:38.376450 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260926 04:06:38.495702 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:38.495392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:36979)
0926 04:06:38.495579 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:38.495656 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":62,"threads_started":1}
I20260926 04:06:39.318470 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:39.318170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:47049)
0926 04:06:39.318349 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:39.318402 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260926 04:06:39.377436 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:39.377166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:40745)
0926 04:06:39.377339 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:39.377387 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260926 04:06:39.496560 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:39.496282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:52711)
0926 04:06:39.496464 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:39.496511 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260926 04:06:39.755021 17741 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:39.755136 17741 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:39.755416 17741 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
W20260926 04:06:39.755908 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
W20260926 04:06:39.756013 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:39.756054 17345 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3eff65f5e3343479049af6691b2dcb6; no voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd
I20260926 04:06:39.756197 17741 raft_consensus.cc:2726] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:06:40.319396 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:40.319135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:47419)
0926 04:06:40.319296 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:40.319345 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260926 04:06:40.378372 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:40.378107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:43887)
0926 04:06:40.378269 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:40.378317 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260926 04:06:40.497488 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:40.497176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:44169)
0926 04:06:40.497375 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:40.497424 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260926 04:06:41.320343 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:41.320085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:38437)
0926 04:06:41.320249 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:41.320296 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260926 04:06:41.379246 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:41.378961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:54957)
0926 04:06:41.379097 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:41.379198 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260926 04:06:41.410611 17747 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:41.410727 17747 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:41.410859 17747 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
W20260926 04:06:41.411435 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
W20260926 04:06:41.411518 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:41.411535 17345 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3eff65f5e3343479049af6691b2dcb6; no voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd
I20260926 04:06:41.411672 17747 raft_consensus.cc:2726] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:06:41.498499 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:41.498164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:55765)
0926 04:06:41.498372 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:41.498451 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":106,"threads_started":1}
I20260926 04:06:42.321136 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:42.320899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:50103)
0926 04:06:42.321046 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:42.321090 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260926 04:06:42.380052 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:42.379810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:51507)
0926 04:06:42.379955 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:42.380003 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260926 04:06:42.499286 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:42.499050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:54995)
0926 04:06:42.499198 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:42.499229 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260926 04:06:43.322031 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:43.321721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:39497)
0926 04:06:43.321922 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:43.321982 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":84,"threads_started":1}
I20260926 04:06:43.380843 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:43.380578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:51617)
0926 04:06:43.380723 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:43.380774 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260926 04:06:43.500051 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:43.499819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:59863)
0926 04:06:43.499959 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:43.500003 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260926 04:06:44.322947 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:44.322686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:43731)
0926 04:06:44.322853 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:44.322899 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260926 04:06:44.381628 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:44.381389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:47617)
0926 04:06:44.381538 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:44.381582 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260926 04:06:44.500789 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:44.500566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:37317)
0926 04:06:44.500690 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:44.500728 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260926 04:06:45.323777 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:45.323509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:52867)
0926 04:06:45.323687 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:45.323728 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260926 04:06:45.382407 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:45.382138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:43867)
0926 04:06:45.382302 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:45.382351 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":61,"threads_started":1}
I20260926 04:06:45.501549 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:45.501323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:58029)
0926 04:06:45.501452 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:45.501486 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260926 04:06:45.543148 17761 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:45.543272 17761 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:45.543766 17761 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
W20260926 04:06:45.543845 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
W20260926 04:06:45.543939 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:45.543975 17345 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3eff65f5e3343479049af6691b2dcb6; no voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd
I20260926 04:06:45.544049 17761 raft_consensus.cc:2726] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:06:46.324609 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:46.324364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:50205)
0926 04:06:46.324516 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:46.324561 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260926 04:06:46.383304 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:46.382992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:36985)
0926 04:06:46.383208 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:46.383258 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
I20260926 04:06:46.502195 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:46.501988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:44285)
0926 04:06:46.502114 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:46.502147 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260926 04:06:47.325484 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:47.325219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:39645)
0926 04:06:47.325381 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:47.325434 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
I20260926 04:06:47.384055 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:47.383819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:43115)
0926 04:06:47.383968 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:47.384007 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260926 04:06:47.502967 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:47.502752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:60879)
0926 04:06:47.502880 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:47.502922 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260926 04:06:48.326330 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:48.326067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:56421)
0926 04:06:48.326229 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:48.326273 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260926 04:06:48.384912 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:48.384621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:37997)
0926 04:06:48.384819 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:48.384862 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":91,"threads_started":1}
I20260926 04:06:48.503640 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:48.503418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:41421)
0926 04:06:48.503548 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:48.503597 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260926 04:06:49.327178 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:49.326908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:34153)
0926 04:06:49.327079 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:49.327130 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260926 04:06:49.385743 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:49.385495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:42613)
0926 04:06:49.385643 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:49.385695 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260926 04:06:49.504395 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:49.504176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:45415)
0926 04:06:49.504303 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:49.504347 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260926 04:06:50.295296 17774 raft_consensus.cc:482] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:50.295393 17774 raft_consensus.cc:504] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a74c0ea83e4e5ca36d2ebbb3bb0acd" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34635 } } peers { permanent_uuid: "09c137d28c984a6e8a6f0cec467b505b" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 35555 } } peers { permanent_uuid: "b3eff65f5e3343479049af6691b2dcb6" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 44489 } }
I20260926 04:06:50.295588 17774 leader_election.cc:288] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635), 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555)
W20260926 04:06:50.296242 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18a74c0ea83e4e5ca36d2ebbb3bb0acd (127.0.200.130:34635): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
W20260926 04:06:50.296384 17345 leader_election.cc:341] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 09c137d28c984a6e8a6f0cec467b505b (127.0.200.131:35555): Not found: Tablet not found: a3f8b119819242d48433fce0ffc2da0e
I20260926 04:06:50.296424 17345 leader_election.cc:302] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3eff65f5e3343479049af6691b2dcb6; no voters: 09c137d28c984a6e8a6f0cec467b505b, 18a74c0ea83e4e5ca36d2ebbb3bb0acd
I20260926 04:06:50.296552 17774 raft_consensus.cc:2726] T a3f8b119819242d48433fce0ffc2da0e P b3eff65f5e3343479049af6691b2dcb6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 04:06:50.327940 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:50.327710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:56325)
0926 04:06:50.327853 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:50.327893 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260926 04:06:50.386718 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:50.386404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:52557)
0926 04:06:50.386621 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:50.386668 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260926 04:06:50.505353 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:50.505087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:47245)
0926 04:06:50.505254 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:50.505304 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260926 04:06:51.328949 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:51.328693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.129:38123)
0926 04:06:51.328853 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:51.328901 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260926 04:06:51.387571 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:51.387347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.130:43739)
0926 04:06:51.387482 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:51.387527 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260926 04:06:51.499941 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17294
I20260926 04:06:51.505075 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17462
I20260926 04:06:51.506924 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:51.506650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:39941 (local address 127.0.200.131:36089)
0926 04:06:51.506817 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:51.506863 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.190:39941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260926 04:06:51.509824 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17595
2026-09-26T04:06:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58252 ms)
[----------] 30 tests from SecurityITest (239686 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:06:51 dist-test-slave-j8c1 krb5kdc[17783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:06:51 dist-test-slave-j8c1 krb5kdc[17783](info): set up 2 sockets
Sep 26 04:06:51 dist-test-slave-j8c1 krb5kdc[17783](info): commencing operation
krb5kdc: starting...
W20260926 04:06:52.558373 802 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:06:52 dist-test-slave-j8c1 krb5kdc[17783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:06:52.685767 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44911
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:44911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:52.766851 17796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:52.767107 17796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:52.767148 17796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:52.768369 17796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:52.768447 17796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:52.768481 17796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:52.768507 17796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:52.768527 17796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:52.770061 17796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:44911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:44911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:52.770341 17796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:52.770641 17796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:52.775615 17804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:52.775661 17801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:52.775700 17796 server_base.cc:1034] running on GCE node
W20260926 04:06:52.775628 17802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:52.776129 17796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:52.776465 17796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:52.777645 17796 hybrid_clock.cc:648] HybridClock initialized: now 1790395612777630 us; error 26 us; skew 500 ppm
Sep 26 04:06:52 dist-test-slave-j8c1 krb5kdc[17783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395612, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:52.780220 17796 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:06:52.780680 17796 webserver.cc:460] Webserver started at http://127.0.200.190:40097/ using document root <none> and password file <none>
I20260926 04:06:52.780921 17796 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:52.780994 17796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:52.781144 17796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:52.782196 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "f5c9a83a37a6475cb6520c0a6f380373"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "433875e9b1e17f45786450fe35c5e1ca"
server_key_iv: "17202f79e5faeba6b673f38673c2ac47"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.782617 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "f5c9a83a37a6475cb6520c0a6f380373"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "433875e9b1e17f45786450fe35c5e1ca"
server_key_iv: "17202f79e5faeba6b673f38673c2ac47"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.783967 17796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:52.784667 17811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:52.784847 17796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:06:52.784989 17796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "f5c9a83a37a6475cb6520c0a6f380373"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "433875e9b1e17f45786450fe35c5e1ca"
server_key_iv: "17202f79e5faeba6b673f38673c2ac47"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.785079 17796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:52.794445 17796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:52.796200 17796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:52.796370 17796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:52.800333 17796 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:44911
I20260926 04:06:52.800372 17863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:44911 every 8 connection(s)
I20260926 04:06:52.800690 17796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:52.801622 17864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:52.803944 17864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: Bootstrap starting.
I20260926 04:06:52.804519 17864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:52.804816 17864 log.cc:824] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:52.805580 17864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: No bootstrap required, opened a new log
I20260926 04:06:52.807404 17864 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } }
I20260926 04:06:52.807554 17864 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:52.807576 17864 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c9a83a37a6475cb6520c0a6f380373, State: Initialized, Role: FOLLOWER
I20260926 04:06:52.807740 17864 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } }
I20260926 04:06:52.807811 17864 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:52.807866 17864 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:52.807920 17864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:52.808468 17864 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } }
I20260926 04:06:52.808554 17864 leader_election.cc:302] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5c9a83a37a6475cb6520c0a6f380373; no voters:
I20260926 04:06:52.808750 17864 leader_election.cc:288] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:52.808853 17867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:52.809020 17864 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:52.809006 17867 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [term 1 LEADER]: Becoming Leader. State: Replica: f5c9a83a37a6475cb6520c0a6f380373, State: Running, Role: LEADER
I20260926 04:06:52.809151 17867 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } }
I20260926 04:06:52.809753 17868 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5c9a83a37a6475cb6520c0a6f380373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } } }
I20260926 04:06:52.809913 17868 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:52.810104 17869 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5c9a83a37a6475cb6520c0a6f380373. Latest consensus state: current_term: 1 leader_uuid: "f5c9a83a37a6475cb6520c0a6f380373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c9a83a37a6475cb6520c0a6f380373" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 44911 } } }
I20260926 04:06:52.810187 17869 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:52.810163 17877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:52.810536 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17796
I20260926 04:06:52.810676 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:52.810776 17877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:52.810886 802 external_mini_cluster.cc:1428] Setting key 69125fc39bcb556f524e7ad41fefcbe0
I20260926 04:06:52.812557 17877 catalog_manager.cc:1347] Generated new cluster ID: 18f51eeadc6646ac83bce22b6bd63965
I20260926 04:06:52.812608 17877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:06:52 dist-test-slave-j8c1 krb5kdc[17783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:52.822947 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:52.811519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.0.200.190:44911)
0926 04:06:52.811875 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:52.811883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:52.811918 (+ 35us) server_negotiation.cc:407] Connection header received
0926 04:06:52.812045 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:52.812052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:52.812115 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:52.812250 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:52.813041 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:52.814522 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:52.815582 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:52.816185 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:52.816298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:52.819100 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:52.819104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:52.819104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:52.819116 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:52.821133 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:52.821526 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:52.821529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:52.821529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:52.821643 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:52.821846 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:52.821847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:52.821848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:52.822176 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:52.822329 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:52.822425 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 04:06:52.822700 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":68,"threads_started":1}
I20260926 04:06:52.831425 17877 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:52.831966 17877 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:52.839097 17877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: Generated new TSK 0
I20260926 04:06:52.839293 17877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:06:52.871309 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:52.952895 17890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:52.953092 17890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:52.953115 17890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:52.954223 17890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:52.954259 17890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:52.954284 17890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:52.955821 17890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:52.956009 17890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:52.956171 17890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:52.961201 17898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:52.961201 17895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:52.961202 17896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:52.961520 17890 server_base.cc:1034] running on GCE node
I20260926 04:06:52.961807 17890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:52.962070 17890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:52.963244 17890 hybrid_clock.cc:648] HybridClock initialized: now 1790395612963226 us; error 28 us; skew 500 ppm
Sep 26 04:06:52 dist-test-slave-j8c1 krb5kdc[17783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395612, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:52.965802 17890 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:06:52.966295 17890 webserver.cc:460] Webserver started at http://127.0.200.129:43015/ using document root <none> and password file <none>
I20260926 04:06:52.966596 17890 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:52.966681 17890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:52.966818 17890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:52.967862 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "d027678d96c34b47970e39dde633bd6e"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "c9ac7c49944c701c573eaf7d7967a068"
server_key_iv: "1cdfb1dbaf9a339fd78d712656751741"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.968230 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "d027678d96c34b47970e39dde633bd6e"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "c9ac7c49944c701c573eaf7d7967a068"
server_key_iv: "1cdfb1dbaf9a339fd78d712656751741"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.969551 17890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:06:52.970335 17905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:52.970638 17890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:52.970745 17890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "d027678d96c34b47970e39dde633bd6e"
format_stamp: "Formatted at 2026-09-26 04:06:52 on dist-test-slave-j8c1"
server_key: "c9ac7c49944c701c573eaf7d7967a068"
server_key_iv: "1cdfb1dbaf9a339fd78d712656751741"
server_key_version: "encryptionkey@0"
I20260926 04:06:52.970837 17890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:52.989758 17890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:52.991951 17890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:52.992137 17890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:52.992396 17890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:52.992766 17890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:52.992833 17890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:52.992877 17890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:52.992902 17890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:52.999229 17890 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:36737
I20260926 04:06:52.999287 18018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:36737 every 8 connection(s)
I20260926 04:06:52.999609 17890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
Sep 26 04:06:52 dist-test-slave-j8c1 krb5kdc[17783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395612, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:53.006399 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 17890
I20260926 04:06:53.006532 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:53.006871 802 external_mini_cluster.cc:1428] Setting key e3865663be665a367d148557534d8a42
I20260926 04:06:53.010460 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.000323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:33689 (local address 127.0.200.190:44911)
0926 04:06:53.000403 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.000405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.000611 (+ 206us) server_negotiation.cc:407] Connection header received
0926 04:06:53.001612 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.001613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.001625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.001677 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.002334 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.002911 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.003924 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.004120 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.004177 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.006751 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.006753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.006754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:53.006763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.008799 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.009384 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.009385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.009385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.009445 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.009747 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.009748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.009748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.009879 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.009941 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.010294 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.010387 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 04:06:53.010762 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.000327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44911 (local address 127.0.200.129:33689)
0926 04:06:53.000502 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.000540 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.001510 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.001800 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.001804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.002058 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:53.002248 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.002251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.003018 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.003019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.003836 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.003839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.004220 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.004221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.004329 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.004891 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:53.004900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:53.006641 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:53.008906 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.008912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.008914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.009279 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.009522 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.009523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.009524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.009666 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.009955 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:53.009961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:53.010144 (+ 183us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.010269 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.010452 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260926 04:06:53.011260 18019 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44911
I20260926 04:06:53.011399 18019 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:53.011611 18019 heartbeater.cc:511] Master 127.0.200.190:44911 requested a full tablet report, sending...
I20260926 04:06:53.012311 17828 ts_manager.cc:194] Registered new tserver with Master: d027678d96c34b47970e39dde633bd6e (127.0.200.129:36737)
I20260926 04:06:53.013325 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:33689
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:06:53.034786 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:53.114509 18024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:53.114797 18024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:53.114821 18024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:53.115998 18024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:53.116041 18024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:53.116067 18024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:53.117565 18024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:53.117826 18024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:53.118072 18024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:53.123081 18029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.123082 18030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:53.123081 18024 server_base.cc:1034] running on GCE node
W20260926 04:06:53.123103 18032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:53.123567 18024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:53.123847 18024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:53.125056 18024 hybrid_clock.cc:648] HybridClock initialized: now 1790395613125050 us; error 58 us; skew 500 ppm
Sep 26 04:06:53 dist-test-slave-j8c1 krb5kdc[17783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:53.127751 18024 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:06:53.128191 18024 webserver.cc:460] Webserver started at http://127.0.200.130:45859/ using document root <none> and password file <none>
I20260926 04:06:53.128389 18024 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:53.128429 18024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:53.128523 18024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:53.129588 18024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "bf9a945a5ca144aeb3002847b58280fa"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fe489e32a38a0a6710f15a12f8ad6948"
server_key_iv: "3c29a0f9d8c9d4d9dcdfc74f58506dff"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.129887 18024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "bf9a945a5ca144aeb3002847b58280fa"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fe489e32a38a0a6710f15a12f8ad6948"
server_key_iv: "3c29a0f9d8c9d4d9dcdfc74f58506dff"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.131325 18024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:06:53.132112 18039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.132390 18024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:53.132485 18024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "bf9a945a5ca144aeb3002847b58280fa"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fe489e32a38a0a6710f15a12f8ad6948"
server_key_iv: "3c29a0f9d8c9d4d9dcdfc74f58506dff"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.132544 18024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:53.147904 18024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:53.150249 18024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:53.150367 18024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:53.150605 18024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:53.150892 18024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:53.150924 18024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.150947 18024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:53.150960 18024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.156764 18024 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:41343
I20260926 04:06:53.156863 18152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:41343 every 8 connection(s)
I20260926 04:06:53.157212 18024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:53.159492 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18024
I20260926 04:06:53.159601 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:53.159799 802 external_mini_cluster.cc:1428] Setting key d462b41889a0204d3adb7038d2874362
Sep 26 04:06:53 dist-test-slave-j8c1 krb5kdc[17783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:06:53.168767 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.158098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:57241 (local address 127.0.200.190:44911)
0926 04:06:53.158183 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.158186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.158361 (+ 175us) server_negotiation.cc:407] Connection header received
0926 04:06:53.159472 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.159474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.159485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.159539 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.160476 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.161047 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.162527 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.162946 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.163040 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.165433 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.165435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.165436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:53.165449 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.167137 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.167633 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.167635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.167635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.167698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.167987 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.167988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.167989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.168151 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.168217 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.168572 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.168664 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 04:06:53.168977 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.158007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44911 (local address 127.0.200.130:57241)
0926 04:06:53.158213 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.158260 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.159329 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.159636 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.159640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.160057 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:53.160333 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.160339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.161154 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.161155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.162407 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.162410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.163047 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.163048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.163153 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.163773 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:53.163782 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:53.165297 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:53.167254 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.167262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.167265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.167523 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.167775 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.167776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.167777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.167902 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.168223 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:53.168229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:53.168409 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.168530 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.168709 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260926 04:06:53.169559 18153 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44911
I20260926 04:06:53.169678 18153 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:53.169868 18153 heartbeater.cc:511] Master 127.0.200.190:44911 requested a full tablet report, sending...
I20260926 04:06:53.170343 17828 ts_manager.cc:194] Registered new tserver with Master: bf9a945a5ca144aeb3002847b58280fa (127.0.200.130:41343)
I20260926 04:06:53.170792 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:57241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:06:53.186057 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:40571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:53.270364 18158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:53.270604 18158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:53.270651 18158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:53.271894 18158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:53.271942 18158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:53.271970 18158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:53.273599 18158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:40571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:53.273859 18158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:53.274147 18158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:53.279103 18166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.279107 18164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.279111 18163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:53.279454 18158 server_base.cc:1034] running on GCE node
I20260926 04:06:53.279685 18158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:53.279963 18158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:53.281104 18158 hybrid_clock.cc:648] HybridClock initialized: now 1790395613281093 us; error 29 us; skew 500 ppm
Sep 26 04:06:53 dist-test-slave-j8c1 krb5kdc[17783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:53.284022 18158 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:06:53.284515 18158 webserver.cc:460] Webserver started at http://127.0.200.131:40239/ using document root <none> and password file <none>
I20260926 04:06:53.284755 18158 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:53.284828 18158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:53.284982 18158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:53.286026 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "390227fc356043edb9b63baf9f41c5e8"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "8ff889d68fb108bfcfcd4f8bdfe02d79"
server_key_iv: "71679869bfd00eca8a9cbff279d49a25"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.286401 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "390227fc356043edb9b63baf9f41c5e8"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "8ff889d68fb108bfcfcd4f8bdfe02d79"
server_key_iv: "71679869bfd00eca8a9cbff279d49a25"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.287901 18158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:53.288717 18173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.288918 18158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:53.289019 18158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "390227fc356043edb9b63baf9f41c5e8"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "8ff889d68fb108bfcfcd4f8bdfe02d79"
server_key_iv: "71679869bfd00eca8a9cbff279d49a25"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.289105 18158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:53.305987 18158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:53.308480 18158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:53.308650 18158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:53.308967 18158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:53.309286 18158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:53.309365 18158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.309401 18158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:53.309445 18158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.315658 18158 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:34141
I20260926 04:06:53.315699 18286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:34141 every 8 connection(s)
I20260926 04:06:53.316025 18158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:53.321549 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18158
I20260926 04:06:53.321645 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:06:53.321854 802 external_mini_cluster.cc:1428] Setting key a5d2a3fca59b2295e5e765a1f5ca0753
Sep 26 04:06:53 dist-test-slave-j8c1 krb5kdc[17783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:53.326700 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.316749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:57675 (local address 127.0.200.190:44911)
0926 04:06:53.316794 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.316796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.317021 (+ 225us) server_negotiation.cc:407] Connection header received
0926 04:06:53.318318 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.318320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.318338 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.318399 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.319188 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.319737 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.320786 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.321001 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.321095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.323990 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.323992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.323993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:53.323999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.325332 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.325731 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.325732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.325732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.325788 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.326078 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.326079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.326079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.326182 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.326231 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.326499 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.326614 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:06:53.327003 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.316712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:44911 (local address 127.0.200.131:57675)
0926 04:06:53.316899 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.316935 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.318101 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.318471 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.318475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.318877 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:53.319073 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.319078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.319847 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.319848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.320662 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.320664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.321105 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.321106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.321244 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.321926 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:53.321935 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:53.323897 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:53.325423 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.325429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.325431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.325654 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.325895 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:53.325896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:53.325896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:53.326008 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:53.326246 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:53.326250 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:53.326379 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.326468 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.326681 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260926 04:06:53.327521 18287 heartbeater.cc:347] Connected to a master server at 127.0.200.190:44911
I20260926 04:06:53.327606 18287 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:53.327822 18287 heartbeater.cc:511] Master 127.0.200.190:44911 requested a full tablet report, sending...
I20260926 04:06:53.328202 17828 ts_manager.cc:194] Registered new tserver with Master: 390227fc356043edb9b63baf9f41c5e8 (127.0.200.131:34141)
I20260926 04:06:53.328548 17828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:57675
I20260926 04:06:53.334326 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:53.342257 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.335929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.0.200.190:44911)
0926 04:06:53.336022 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.336025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.336092 (+ 67us) server_negotiation.cc:407] Connection header received
0926 04:06:53.336160 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.336161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.336172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.336226 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.336872 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.337388 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.338155 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.338364 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.338457 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.339881 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.339882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.339882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:53.339889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.341025 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.341406 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.341407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.341408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.341471 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:53.341681 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:53.341682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:53.341682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:53.341846 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.341969 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.342069 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.342152 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 04:06:53.343715 17828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:53.345232 17828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:53.352177 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.347718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:34141 (local address 127.0.0.1:55456)
0926 04:06:53.347968 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.347977 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.348020 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.348496 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.348500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.348502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:53.348575 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.348587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.350054 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.350055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.351199 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.351201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.351880 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.351957 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.351970 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.352035 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.352088 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":29,"thread_start_us":85,"threads_started":1}
I20260926 04:06:53.352279 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.347775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.0.200.129:36737)
0926 04:06:53.348643 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.348647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.348657 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:53.348695 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.348697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.348711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.348788 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:53.349040 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.349726 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351238 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.351943 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.352003 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.352052 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.352145 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.352191 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":93,"threads_started":1}
I20260926 04:06:53.352411 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.348144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.0.200.130:41343)
0926 04:06:53.348510 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.348514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.348525 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:53.348562 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.348564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.348575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.348709 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:53.348795 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.349944 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351166 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.351867 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351939 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.351995 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.352282 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.352331 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":64,"threads_started":1}
I20260926 04:06:53.352521 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.347940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.0.200.131:34141)
0926 04:06:53.348297 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.348302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.348310 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:06:53.348342 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.348344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.348356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.348486 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:53.348884 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.349925 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351291 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.351770 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.351878 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.351939 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.352406 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.352452 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":76,"threads_started":1}
I20260926 04:06:53.352820 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.347676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:36737 (local address 127.0.0.1:54344)
0926 04:06:53.347828 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.347868 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.347937 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.348915 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.348917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.348919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:53.348954 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.348955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.349843 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.349844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.351161 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.351164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.352016 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.352017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.352072 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.352083 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.352132 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.352166 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":48,"threads_started":1}
I20260926 04:06:53.352821 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.347713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:41343 (local address 127.0.0.1:54074)
0926 04:06:53.347943 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.347955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.348009 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.348701 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.348701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.348703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:53.348748 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.348749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.350056 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.350057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.351053 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.351056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.352032 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.352033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.352105 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.352109 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.352171 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.352210 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":17,"thread_start_us":100,"threads_started":1}
I20260926 04:06:53.353305 18221 tablet_service.cc:1467] Processing CreateTablet for tablet a99dd5c342474fa28436cf89ae830074 (DEFAULT_TABLE table=test-table [id=eb8b72967e8743f98ded0283ccf6d892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:53.353432 18087 tablet_service.cc:1467] Processing CreateTablet for tablet a99dd5c342474fa28436cf89ae830074 (DEFAULT_TABLE table=test-table [id=eb8b72967e8743f98ded0283ccf6d892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:53.353699 18221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99dd5c342474fa28436cf89ae830074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:53.353817 18087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99dd5c342474fa28436cf89ae830074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:53.355183 17953 tablet_service.cc:1467] Processing CreateTablet for tablet a99dd5c342474fa28436cf89ae830074 (DEFAULT_TABLE table=test-table [id=eb8b72967e8743f98ded0283ccf6d892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:53.355505 17953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99dd5c342474fa28436cf89ae830074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:53.356689 18304 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Bootstrap starting.
I20260926 04:06:53.357064 18305 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Bootstrap starting.
I20260926 04:06:53.357362 18304 tablet_bootstrap.cc:654] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:53.357715 18305 tablet_bootstrap.cc:654] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:53.357941 18304 log.cc:824] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:53.358249 18305 log.cc:824] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:53.358741 18304 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: No bootstrap required, opened a new log
I20260926 04:06:53.358814 18304 ts_tablet_manager.cc:1397] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.358985 18305 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: No bootstrap required, opened a new log
I20260926 04:06:53.359050 18305 ts_tablet_manager.cc:1397] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.360342 18304 raft_consensus.cc:348] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.360514 18304 raft_consensus.cc:374] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:53.360562 18304 raft_consensus.cc:729] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390227fc356043edb9b63baf9f41c5e8, State: Initialized, Role: FOLLOWER
I20260926 04:06:53.360466 18305 raft_consensus.cc:348] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.360647 18305 raft_consensus.cc:374] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:53.360689 18305 raft_consensus.cc:729] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9a945a5ca144aeb3002847b58280fa, State: Initialized, Role: FOLLOWER
I20260926 04:06:53.360713 18304 consensus_queue.cc:260] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.360798 18305 consensus_queue.cc:260] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.361002 18287 heartbeater.cc:503] Master 127.0.200.190:44911 was elected leader, sending a full tablet report...
I20260926 04:06:53.361121 18305 ts_tablet_manager.cc:1428] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.361146 18153 heartbeater.cc:503] Master 127.0.200.190:44911 was elected leader, sending a full tablet report...
I20260926 04:06:53.360992 18304 ts_tablet_manager.cc:1428] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.361323 18308 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Bootstrap starting.
I20260926 04:06:53.361963 18308 tablet_bootstrap.cc:654] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:53.362540 18308 log.cc:824] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:53.363301 18308 tablet_bootstrap.cc:492] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: No bootstrap required, opened a new log
I20260926 04:06:53.363430 18308 ts_tablet_manager.cc:1397] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.364918 18308 raft_consensus.cc:348] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.365063 18308 raft_consensus.cc:374] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:53.365087 18308 raft_consensus.cc:729] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d027678d96c34b47970e39dde633bd6e, State: Initialized, Role: FOLLOWER
I20260926 04:06:53.365217 18308 consensus_queue.cc:260] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.365516 18308 ts_tablet_manager.cc:1428] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:53.365526 18019 heartbeater.cc:503] Master 127.0.200.190:44911 was elected leader, sending a full tablet report...
W20260926 04:06:53.407908 18154 tablet.cc:2369] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:06:53.500406 18020 tablet.cc:2369] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:53.539693 18310 raft_consensus.cc:482] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:53.539887 18310 raft_consensus.cc:504] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.540323 18310 leader_election.cc:288] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d027678d96c34b47970e39dde633bd6e (127.0.200.129:36737), 390227fc356043edb9b63baf9f41c5e8 (127.0.200.131:34141)
I20260926 04:06:53.543799 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.540521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:36737 (local address 127.0.200.130:49665)
0926 04:06:53.540606 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.540619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.540675 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.540854 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.540855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.540857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:53.540930 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.540932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.541737 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.541738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.542713 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.542716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.543492 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.543493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.543571 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.543575 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.543663 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.543704 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 04:06:53.543869 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.540633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:49665 (local address 127.0.200.129:36737)
0926 04:06:53.540708 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.540710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.540726 (+ 16us) server_negotiation.cc:407] Connection header received
0926 04:06:53.540758 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.540759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.540767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.540818 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:53.541020 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.541631 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.542904 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.543398 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.543462 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.543470 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.543707 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.543797 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 04:06:53.544128 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.540641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:33491 (local address 127.0.200.131:34141)
0926 04:06:53.540686 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.540688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.541014 (+ 326us) server_negotiation.cc:407] Connection header received
0926 04:06:53.541110 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.541111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.541119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.541164 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:53.541379 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.541992 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.543225 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.543713 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.543780 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.543787 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.544012 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.544057 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:06:53.544196 17973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99dd5c342474fa28436cf89ae830074" candidate_uuid: "bf9a945a5ca144aeb3002847b58280fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d027678d96c34b47970e39dde633bd6e" is_pre_election: true
I20260926 04:06:53.544373 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.540530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:34141 (local address 127.0.200.130:33491)
0926 04:06:53.540924 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.540933 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.541010 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.541187 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.541188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.541190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:53.541263 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.541265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.542119 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.542120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.543132 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.543135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.543829 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.543830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.543903 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.543908 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.544234 (+ 326us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.544288 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":169,"thread_start_us":131,"threads_started":1}
I20260926 04:06:53.544375 17973 raft_consensus.cc:2454] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf9a945a5ca144aeb3002847b58280fa in term 0.
I20260926 04:06:53.544688 18043 leader_election.cc:302] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf9a945a5ca144aeb3002847b58280fa, d027678d96c34b47970e39dde633bd6e; no voters:
I20260926 04:06:53.544811 18241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99dd5c342474fa28436cf89ae830074" candidate_uuid: "bf9a945a5ca144aeb3002847b58280fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390227fc356043edb9b63baf9f41c5e8" is_pre_election: true
I20260926 04:06:53.544848 18310 raft_consensus.cc:2781] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:53.544919 18310 raft_consensus.cc:482] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:53.544952 18310 raft_consensus.cc:3037] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:53.544976 18241 raft_consensus.cc:2454] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf9a945a5ca144aeb3002847b58280fa in term 0.
I20260926 04:06:53.545701 18310 raft_consensus.cc:504] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.545866 18310 leader_election.cc:288] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [CANDIDATE]: Term 1 election: Requested vote from peers d027678d96c34b47970e39dde633bd6e (127.0.200.129:36737), 390227fc356043edb9b63baf9f41c5e8 (127.0.200.131:34141)
I20260926 04:06:53.546051 18241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99dd5c342474fa28436cf89ae830074" candidate_uuid: "bf9a945a5ca144aeb3002847b58280fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390227fc356043edb9b63baf9f41c5e8"
I20260926 04:06:53.546093 17973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99dd5c342474fa28436cf89ae830074" candidate_uuid: "bf9a945a5ca144aeb3002847b58280fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d027678d96c34b47970e39dde633bd6e"
I20260926 04:06:53.546137 18241 raft_consensus.cc:3037] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:53.546200 17973 raft_consensus.cc:3037] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:53.546985 17973 raft_consensus.cc:2454] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf9a945a5ca144aeb3002847b58280fa in term 1.
I20260926 04:06:53.546993 18241 raft_consensus.cc:2454] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf9a945a5ca144aeb3002847b58280fa in term 1.
I20260926 04:06:53.547163 18043 leader_election.cc:302] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf9a945a5ca144aeb3002847b58280fa, d027678d96c34b47970e39dde633bd6e; no voters:
I20260926 04:06:53.547253 18310 raft_consensus.cc:2781] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:53.547395 18310 raft_consensus.cc:686] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 1 LEADER]: Becoming Leader. State: Replica: bf9a945a5ca144aeb3002847b58280fa, State: Running, Role: LEADER
I20260926 04:06:53.547511 18310 consensus_queue.cc:237] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } }
I20260926 04:06:53.548390 17828 catalog_manager.cc:5495] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa reported cstate change: term changed from 0 to 1, leader changed from <none> to bf9a945a5ca144aeb3002847b58280fa (127.0.200.130). New cstate: current_term: 1 leader_uuid: "bf9a945a5ca144aeb3002847b58280fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf9a945a5ca144aeb3002847b58280fa" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 } health_report { overall_health: UNKNOWN } } }
W20260926 04:06:53.566689 18288 tablet.cc:2369] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:53.580366 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.577007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.0.200.130:41343)
0926 04:06:53.577095 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.577097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.577104 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:06:53.577146 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.577147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.577171 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.577246 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:53.577368 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.577857 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.578999 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.579349 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.579438 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.579666 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:53.579808 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:53.579919 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.580205 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.580258 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 04:06:53.582425 17973 raft_consensus.cc:1261] T a99dd5c342474fa28436cf89ae830074 P d027678d96c34b47970e39dde633bd6e [term 1 FOLLOWER]: Refusing update from remote peer bf9a945a5ca144aeb3002847b58280fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:53.582448 18241 raft_consensus.cc:1261] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 1 FOLLOWER]: Refusing update from remote peer bf9a945a5ca144aeb3002847b58280fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:53.582844 18314 consensus_queue.cc:1035] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d027678d96c34b47970e39dde633bd6e" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 36737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:53.583012 18310 consensus_queue.cc:1035] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "390227fc356043edb9b63baf9f41c5e8" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:53.585381 18317 mvcc.cc:205] Tried to move back new op lower bound from 7333460433231478784 to 7333460433091964928. Current Snapshot: MvccSnapshot[applied={T|T < 7333460433231478784}]
I20260926 04:06:53.585486 18318 mvcc.cc:205] Tried to move back new op lower bound from 7333460433231478784 to 7333460433091964928. Current Snapshot: MvccSnapshot[applied={T|T < 7333460433231478784}]
I20260926 04:06:53.585667 18319 mvcc.cc:205] Tried to move back new op lower bound from 7333460433231478784 to 7333460433091964928. Current Snapshot: MvccSnapshot[applied={T|T < 7333460433231478784}]
I20260926 04:06:53.589185 17827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:53.589305 17827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:53.590154 17827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5c9a83a37a6475cb6520c0a6f380373: Sending DeleteTablet for 3 replicas of tablet a99dd5c342474fa28436cf89ae830074
I20260926 04:06:53.590631 17953 tablet_service.cc:1514] Processing DeleteTablet for tablet a99dd5c342474fa28436cf89ae830074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:53 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:54344
I20260926 04:06:53.590680 18087 tablet_service.cc:1514] Processing DeleteTablet for tablet a99dd5c342474fa28436cf89ae830074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:53 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:54074
I20260926 04:06:53.590972 18221 tablet_service.cc:1514] Processing DeleteTablet for tablet a99dd5c342474fa28436cf89ae830074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:53 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260926 04:06:53.591097 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17890
I20260926 04:06:53.591727 18329 tablet_replica.cc:330] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: stopping tablet replica
I20260926 04:06:53.591893 18328 tablet_replica.cc:330] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: stopping tablet replica
I20260926 04:06:53.591881 18329 raft_consensus.cc:2229] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:53.592005 18328 raft_consensus.cc:2229] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:53.592142 18328 raft_consensus.cc:2258] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:53.592350 18329 raft_consensus.cc:2258] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:53.592610 18328 ts_tablet_manager.cc:1905] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:53.592923 18329 ts_tablet_manager.cc:1905] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:53.594882 18328 ts_tablet_manager.cc:1918] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:53.594969 18328 log.cc:1196] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/a99dd5c342474fa28436cf89ae830074
I20260926 04:06:53.595153 18329 ts_tablet_manager.cc:1918] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:53.595237 18328 ts_tablet_manager.cc:1939] T a99dd5c342474fa28436cf89ae830074 P bf9a945a5ca144aeb3002847b58280fa: Deleting consensus metadata
I20260926 04:06:53.595341 18329 log.cc:1196] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/a99dd5c342474fa28436cf89ae830074
I20260926 04:06:53.595691 18329 ts_tablet_manager.cc:1939] T a99dd5c342474fa28436cf89ae830074 P 390227fc356043edb9b63baf9f41c5e8: Deleting consensus metadata
I20260926 04:06:53.596226 17812 catalog_manager.cc:4841] TS 390227fc356043edb9b63baf9f41c5e8 (127.0.200.131:34141): tablet a99dd5c342474fa28436cf89ae830074 (table test-table [id=eb8b72967e8743f98ded0283ccf6d892]) successfully deleted
I20260926 04:06:53.596720 17813 catalog_manager.cc:4841] TS bf9a945a5ca144aeb3002847b58280fa (127.0.200.130:41343): tablet a99dd5c342474fa28436cf89ae830074 (table test-table [id=eb8b72967e8743f98ded0283ccf6d892]) successfully deleted
W20260926 04:06:53.600030 17815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:36737 (error 108)
I20260926 04:06:53.600481 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18024
I20260926 04:06:53.601156 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.600639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:36737 (local address 127.0.0.1:54348)
0926 04:06:53.601069 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.601123 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:36737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407}
W20260926 04:06:53.601289 17815 catalog_manager.cc:4584] TS d027678d96c34b47970e39dde633bd6e (127.0.200.129:36737): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a99dd5c342474fa28436cf89ae830074: Network error: Client connection negotiation failed: client connection to 127.0.200.129:36737: connect: Connection refused (error 111)
I20260926 04:06:53.605352 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18158
I20260926 04:06:53.610280 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 17796
2026-09-26T04:06:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2100 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T04:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:53Z Disabled control of system clock
I20260926 04:06:53.639164 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:35797
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:35797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:53.720189 18334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:53.720419 18334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:53.720451 18334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:53.721558 18334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:53.721596 18334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:53.721608 18334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:53.721622 18334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:53.721634 18334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:53.722944 18334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:35797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:35797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:53.723253 18334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:53.723564 18334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:53.728540 18342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.728590 18339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.728549 18340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:53.728863 18334 server_base.cc:1034] running on GCE node
I20260926 04:06:53.729118 18334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:53.729363 18334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:53.730633 18334 hybrid_clock.cc:648] HybridClock initialized: now 1790395613730609 us; error 32 us; skew 500 ppm
I20260926 04:06:53.732105 18334 webserver.cc:460] Webserver started at http://127.0.200.190:42965/ using document root <none> and password file <none>
I20260926 04:06:53.732410 18334 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:53.732528 18334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:53.732684 18334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:53.733810 18334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "a70dfdccfa8a4190baf323b1788af120"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "2fc18d5b71677b782f3b886f205e4f88"
server_key_iv: "2dd2df613faabb7c3fd844fedb720595"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.734205 18334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "a70dfdccfa8a4190baf323b1788af120"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "2fc18d5b71677b782f3b886f205e4f88"
server_key_iv: "2dd2df613faabb7c3fd844fedb720595"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.735719 18334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:06:53.736513 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.736693 18334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:53.736796 18334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "a70dfdccfa8a4190baf323b1788af120"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "2fc18d5b71677b782f3b886f205e4f88"
server_key_iv: "2dd2df613faabb7c3fd844fedb720595"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.736881 18334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:53.774636 18334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:53.775027 18334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:53.775211 18334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:53.779484 18334 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:35797
I20260926 04:06:53.779548 18400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:35797 every 8 connection(s)
I20260926 04:06:53.780025 18334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:53.781270 18401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:53.783681 18401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: Bootstrap starting.
I20260926 04:06:53.784080 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18334
I20260926 04:06:53.784194 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:53.784401 18401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:53.784418 802 external_mini_cluster.cc:1428] Setting key 05eba7715b4d51520511a2450a7465a2
I20260926 04:06:53.784741 18401 log.cc:824] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:53.785444 18401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: No bootstrap required, opened a new log
I20260926 04:06:53.787447 18401 raft_consensus.cc:348] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } }
I20260926 04:06:53.787616 18401 raft_consensus.cc:374] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:53.787662 18401 raft_consensus.cc:729] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a70dfdccfa8a4190baf323b1788af120, State: Initialized, Role: FOLLOWER
I20260926 04:06:53.787817 18401 consensus_queue.cc:260] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } }
I20260926 04:06:53.787922 18401 raft_consensus.cc:388] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:53.787973 18401 raft_consensus.cc:482] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:53.788029 18401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:53.788589 18401 raft_consensus.cc:504] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } }
I20260926 04:06:53.788715 18401 leader_election.cc:302] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a70dfdccfa8a4190baf323b1788af120; no voters:
I20260926 04:06:53.788892 18401 leader_election.cc:288] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:53.788960 18406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:53.789099 18406 raft_consensus.cc:686] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [term 1 LEADER]: Becoming Leader. State: Replica: a70dfdccfa8a4190baf323b1788af120, State: Running, Role: LEADER
I20260926 04:06:53.789184 18401 sys_catalog.cc:564] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:53.789197 18406 consensus_queue.cc:237] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } }
I20260926 04:06:53.790037 18407 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a70dfdccfa8a4190baf323b1788af120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } } }
I20260926 04:06:53.790109 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.785000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.0.200.190:35797)
0926 04:06:53.785227 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.785235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.785269 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:06:53.785750 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.785757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.785815 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.785978 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.786200 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.787143 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.788096 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.788511 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.788659 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.789346 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.789351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.789351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:53.789581 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.789622 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:53.789637 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.789698 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.789803 (+ 105us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.789904 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260926 04:06:53.790145 18408 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a70dfdccfa8a4190baf323b1788af120. Latest consensus state: current_term: 1 leader_uuid: "a70dfdccfa8a4190baf323b1788af120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dfdccfa8a4190baf323b1788af120" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 35797 } } }
I20260926 04:06:53.790189 18408 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:53.790375 18415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:53.790120 18407 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:53.790876 18415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:53.792572 18415 catalog_manager.cc:1347] Generated new cluster ID: 39dd68b8941b42efb414c89b15912c4b
I20260926 04:06:53.792624 18415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:06:53.801193 18415 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:53.801707 18415 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:53.814332 18415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: Generated new TSK 0
I20260926 04:06:53.814632 18415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:06:53.818686 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--builtin_ntp_servers=127.0.200.148:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:53.915515 18425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:53.915776 18425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:53.915819 18425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:53.917092 18425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:53.917181 18425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:53.917227 18425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:53.918802 18425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:53.919116 18425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:53.919469 18425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:53.924770 18430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.924775 18431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:53.925029 18433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:53.925042 18425 server_base.cc:1034] running on GCE node
I20260926 04:06:53.925269 18425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:53.925544 18425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:53.926729 18425 hybrid_clock.cc:648] HybridClock initialized: now 1790395613926712 us; error 33 us; skew 500 ppm
I20260926 04:06:53.928159 18425 webserver.cc:460] Webserver started at http://127.0.200.129:32805/ using document root <none> and password file <none>
I20260926 04:06:53.928467 18425 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:53.928550 18425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:53.928707 18425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:53.929869 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "2c0f73f325af46babdf304fea3a0f6da"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fa0657d182ed5955765cb9fb0020011b"
server_key_iv: "28ea1c5a9cbff2a1a44738fa1b2ae1b3"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.930271 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "2c0f73f325af46babdf304fea3a0f6da"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fa0657d182ed5955765cb9fb0020011b"
server_key_iv: "28ea1c5a9cbff2a1a44738fa1b2ae1b3"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.931888 18425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:06:53.932925 18439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.933194 18425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:53.933311 18425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "2c0f73f325af46babdf304fea3a0f6da"
format_stamp: "Formatted at 2026-09-26 04:06:53 on dist-test-slave-j8c1"
server_key: "fa0657d182ed5955765cb9fb0020011b"
server_key_iv: "28ea1c5a9cbff2a1a44738fa1b2ae1b3"
server_key_version: "encryptionkey@0"
I20260926 04:06:53.933408 18425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:53.955642 18425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:53.955981 18425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:53.956144 18425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:53.956388 18425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:53.956719 18425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:53.956780 18425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.956823 18425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:53.956847 18425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:53.962749 18425 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:40515
I20260926 04:06:53.962810 18552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:40515 every 8 connection(s)
I20260926 04:06:53.963130 18425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:53.964438 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18425
I20260926 04:06:53.964529 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:53.964718 802 external_mini_cluster.cc:1428] Setting key d02c7dfba8c7737f5c7693d12a0a2b31
I20260926 04:06:53.965672 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--builtin_ntp_servers=127.0.200.148:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 04:06:53.968799 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.963832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:39561 (local address 127.0.200.190:35797)
0926 04:06:53.963882 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:53.963884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:53.964103 (+ 219us) server_negotiation.cc:407] Connection header received
0926 04:06:53.964810 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:53.964811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:53.964825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:53.964949 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:53.965673 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.966230 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.967325 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.967527 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.967614 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.968543 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:53.968544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:53.968544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:53.968548 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:53.968564 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:53.968568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:53.968613 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:53.968717 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 04:06:53.968740 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:06:53.969158 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:53.963787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:35797 (local address 127.0.200.129:39561)
0926 04:06:53.963968 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:53.964009 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:53.964676 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:53.965036 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:53.965040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:53.965051 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:53.965568 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.965572 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.966337 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.966338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.967226 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:53.967228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:53.967623 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:53.967624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:53.967784 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:53.968347 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:53.968348 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:53.968387 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:53.968425 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:53.968635 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:53.968641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:53.968644 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:53.968712 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:06:53.968829 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260926 04:06:53.969708 18553 heartbeater.cc:347] Connected to a master server at 127.0.200.190:35797
I20260926 04:06:53.969797 18553 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:53.969991 18553 heartbeater.cc:511] Master 127.0.200.190:35797 requested a full tablet report, sending...
I20260926 04:06:53.970499 18365 ts_manager.cc:194] Registered new tserver with Master: 2c0f73f325af46babdf304fea3a0f6da (127.0.200.129:40515)
I20260926 04:06:53.971166 18365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.129:39561
W20260926 04:06:54.047747 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:54.048004 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:54.048043 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:54.049222 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:54.049296 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:54.049340 18556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:54.050789 18556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:54.051077 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:54.051409 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:54.056838 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:54.056831 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:54.056831 18564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:54.057111 18556 server_base.cc:1034] running on GCE node
I20260926 04:06:54.057345 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:54.057572 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:54.058755 18556 hybrid_clock.cc:648] HybridClock initialized: now 1790395614058721 us; error 31 us; skew 500 ppm
I20260926 04:06:54.060212 18556 webserver.cc:460] Webserver started at http://127.0.200.130:44005/ using document root <none> and password file <none>
I20260926 04:06:54.060526 18556 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:54.060578 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:54.060722 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:54.061915 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "629833be2d714af389a73518d03c8960"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "ad098e2dd38c3648adf06fbe393794b0"
server_key_iv: "78b55c27be42beec89797e7de4e6ccb5"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.062309 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "629833be2d714af389a73518d03c8960"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "ad098e2dd38c3648adf06fbe393794b0"
server_key_iv: "78b55c27be42beec89797e7de4e6ccb5"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.063836 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:06:54.064759 18570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.065043 18556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:54.065157 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "629833be2d714af389a73518d03c8960"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "ad098e2dd38c3648adf06fbe393794b0"
server_key_iv: "78b55c27be42beec89797e7de4e6ccb5"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.065249 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:54.080196 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:54.080619 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:54.080812 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:54.081103 18556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:54.081596 18556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:54.081701 18556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.081750 18556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:54.081771 18556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.088176 18556 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:41423
I20260926 04:06:54.088214 18683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:41423 every 8 connection(s)
I20260926 04:06:54.088613 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:54.091351 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18556
I20260926 04:06:54.091464 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:54.091676 802 external_mini_cluster.cc:1428] Setting key 8723a407f9a61c6287da4594131dbe9a
I20260926 04:06:54.092720 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--builtin_ntp_servers=127.0.200.148:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 04:06:54.095268 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.089534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:47063 (local address 127.0.200.190:35797)
0926 04:06:54.089593 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.089595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.089867 (+ 272us) server_negotiation.cc:407] Connection header received
0926 04:06:54.090454 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.090456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.090468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.090526 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:54.091543 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.092197 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.093552 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.093784 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.093893 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.094926 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:54.094927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:54.094927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:54.094934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:54.094950 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:54.094955 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:54.095040 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.095154 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.095185 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:06:54.095551 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.089500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:35797 (local address 127.0.200.130:47063)
0926 04:06:54.089746 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.089783 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.090291 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.090661 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.090666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.090676 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:54.091406 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.091414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.092386 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.092387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.093438 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.093440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.093906 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.093907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.094054 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.094734 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:54.094735 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:54.094765 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:54.094789 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:54.095040 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:54.095047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:54.095050 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.095155 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.095268 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260926 04:06:54.096107 18684 heartbeater.cc:347] Connected to a master server at 127.0.200.190:35797
I20260926 04:06:54.096242 18684 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:54.096446 18684 heartbeater.cc:511] Master 127.0.200.190:35797 requested a full tablet report, sending...
I20260926 04:06:54.096946 18365 ts_manager.cc:194] Registered new tserver with Master: 629833be2d714af389a73518d03c8960 (127.0.200.130:41423)
I20260926 04:06:54.097541 18365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.130:47063
W20260926 04:06:54.178792 18687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:54.179004 18687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:54.179028 18687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:54.180222 18687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:54.180266 18687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:54.180294 18687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:54.181795 18687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:54.182003 18687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:54.182267 18687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:06:54.187754 18687 server_base.cc:1034] running on GCE node
W20260926 04:06:54.187718 18692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:54.187713 18693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:54.187718 18695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:54.188194 18687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:54.188474 18687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:54.189842 18687 hybrid_clock.cc:648] HybridClock initialized: now 1790395614189831 us; error 45 us; skew 500 ppm
I20260926 04:06:54.191375 18687 webserver.cc:460] Webserver started at http://127.0.200.131:33417/ using document root <none> and password file <none>
I20260926 04:06:54.191648 18687 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:54.191722 18687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:54.191903 18687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:54.193048 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "00c3752c1d514cc7972ff931f00955c7"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "fa5afe4a97157565b25f24264b652e41"
server_key_iv: "cb90d5389890f0a9c7731b809bb76a54"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.193499 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "00c3752c1d514cc7972ff931f00955c7"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "fa5afe4a97157565b25f24264b652e41"
server_key_iv: "cb90d5389890f0a9c7731b809bb76a54"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.194934 18687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:06:54.195845 18701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.196131 18687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:54.196259 18687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "00c3752c1d514cc7972ff931f00955c7"
format_stamp: "Formatted at 2026-09-26 04:06:54 on dist-test-slave-j8c1"
server_key: "fa5afe4a97157565b25f24264b652e41"
server_key_iv: "cb90d5389890f0a9c7731b809bb76a54"
server_key_version: "encryptionkey@0"
I20260926 04:06:54.196362 18687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:54.217772 18687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:54.218130 18687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:54.218295 18687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:54.218650 18687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:54.219003 18687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:54.219064 18687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.219108 18687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:54.219134 18687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:54.224958 18687 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:46761
I20260926 04:06:54.225037 18814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:46761 every 8 connection(s)
I20260926 04:06:54.225395 18687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:54.228569 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18687
I20260926 04:06:54.228689 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:06:54.228947 802 external_mini_cluster.cc:1428] Setting key d070d460bd3f5f4f98750e0c614f046b
I20260926 04:06:54.232106 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.226208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:50255 (local address 127.0.200.190:35797)
0926 04:06:54.226282 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.226284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.226533 (+ 249us) server_negotiation.cc:407] Connection header received
0926 04:06:54.227180 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.227181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.227195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.227259 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:54.227979 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.228640 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.230203 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.230420 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.230490 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.231751 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:54.231753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:54.231753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:54.231757 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:54.231773 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:54.231778 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:54.231838 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.232002 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.232029 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:06:54.232477 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.226188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:35797 (local address 127.0.200.131:50255)
0926 04:06:54.226400 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.226441 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.227043 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.227337 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.227341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.227349 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:54.227870 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.227875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.228836 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.228838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.230099 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.230102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.230534 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.230535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.230845 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.231576 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:54.231576 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:54.231608 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:54.231633 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:54.231879 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:54.231886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:54.231889 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.231966 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.232067 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260926 04:06:54.233165 18815 heartbeater.cc:347] Connected to a master server at 127.0.200.190:35797
I20260926 04:06:54.233264 18815 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:54.233425 18815 heartbeater.cc:511] Master 127.0.200.190:35797 requested a full tablet report, sending...
I20260926 04:06:54.233845 18365 ts_manager.cc:194] Registered new tserver with Master: 00c3752c1d514cc7972ff931f00955c7 (127.0.200.131:46761)
I20260926 04:06:54.234244 18365 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.131:50255
I20260926 04:06:54.241483 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:54.246447 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.243166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.0.200.190:35797)
0926 04:06:54.243260 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.243262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.243411 (+ 149us) server_negotiation.cc:407] Connection header received
0926 04:06:54.243509 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.243510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.243521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.243599 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:54.243787 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.244291 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.245044 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.245243 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.245353 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.246057 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:54.246058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:54.246059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:54.246064 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:54.246085 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:54.246089 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:54.246155 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.246317 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.246343 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 04:06:54.249347 18365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:54.251140 18365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:54.257893 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.0.200.129:40515)
0926 04:06:54.254072 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.254076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.254294 (+ 218us) server_negotiation.cc:407] Connection header received
0926 04:06:54.254337 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.254339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.254357 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.254729 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:54.254798 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255849 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.256939 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257458 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257560 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.257662 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.257760 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.257797 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260926 04:06:54.257881 18824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:40515 (local address 127.0.0.1:36654)
0926 04:06:54.254133 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.254143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.254234 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.254590 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.254591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.254593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:54.254643 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.254645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255941 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.255942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.256846 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.256848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257561 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.257641 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.257654 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.257735 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.257796 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":122,"threads_started":1}
I20260926 04:06:54.258157 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:46761 (local address 127.0.0.1:50678)
0926 04:06:54.254127 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.254137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.254201 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.254407 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.254408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.254409 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:54.254443 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.254444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255822 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.255822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.256901 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.256903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257886 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.257975 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.257979 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.258038 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.258089 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":82,"threads_started":1}
I20260926 04:06:54.258416 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.0.200.131:46761)
0926 04:06:54.254072 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.254076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.254264 (+ 188us) server_negotiation.cc:407] Connection header received
0926 04:06:54.254314 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.254316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.254332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.254412 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:54.254501 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255719 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.256992 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257685 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257755 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.257845 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.258261 (+ 416us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.258313 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260926 04:06:54.258504 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:41423 (local address 127.0.0.1:38888)
0926 04:06:54.253840 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.253879 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.253998 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.254328 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.254333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.254336 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:54.254409 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.254410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255488 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.255489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.256841 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.256844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257886 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.258329 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.258334 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.258387 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.258431 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":62,"thread_start_us":59,"threads_started":1}
I20260926 04:06:54.258735 18487 tablet_service.cc:1467] Processing CreateTablet for tablet 412e686a1201409e96183cc2dd264b0e (DEFAULT_TABLE table=test-table [id=dd285a7352184146a7eb6f0cd5277272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:54.258922 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.253665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.0.200.130:41423)
0926 04:06:54.253822 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.253827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.253975 (+ 148us) server_negotiation.cc:407] Connection header received
0926 04:06:54.254077 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.254080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.254095 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.254165 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:54.254529 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.255372 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.256948 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.257682 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.257755 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.257845 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.258799 (+ 954us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.258842 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260926 04:06:54.259136 18487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412e686a1201409e96183cc2dd264b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:54.259084 18749 tablet_service.cc:1467] Processing CreateTablet for tablet 412e686a1201409e96183cc2dd264b0e (DEFAULT_TABLE table=test-table [id=dd285a7352184146a7eb6f0cd5277272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:54.259419 18749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412e686a1201409e96183cc2dd264b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:54.259479 18618 tablet_service.cc:1467] Processing CreateTablet for tablet 412e686a1201409e96183cc2dd264b0e (DEFAULT_TABLE table=test-table [id=dd285a7352184146a7eb6f0cd5277272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:54.259792 18618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412e686a1201409e96183cc2dd264b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:54.262245 18829 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Bootstrap starting.
I20260926 04:06:54.262890 18830 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Bootstrap starting.
I20260926 04:06:54.263013 18829 tablet_bootstrap.cc:654] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:54.263139 18831 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Bootstrap starting.
I20260926 04:06:54.263340 18829 log.cc:824] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:54.263727 18830 tablet_bootstrap.cc:654] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:54.263811 18831 tablet_bootstrap.cc:654] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:54.264063 18830 log.cc:824] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:54.264108 18831 log.cc:824] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:54.264235 18829 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: No bootstrap required, opened a new log
I20260926 04:06:54.264314 18829 ts_tablet_manager.cc:1397] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:06:54.264820 18830 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: No bootstrap required, opened a new log
I20260926 04:06:54.264879 18831 tablet_bootstrap.cc:492] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: No bootstrap required, opened a new log
I20260926 04:06:54.264912 18830 ts_tablet_manager.cc:1397] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:06:54.264954 18831 ts_tablet_manager.cc:1397] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:06:54.267010 18830 raft_consensus.cc:348] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.267057 18829 raft_consensus.cc:348] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.267184 18830 raft_consensus.cc:374] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:54.267226 18829 raft_consensus.cc:374] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:54.267263 18829 raft_consensus.cc:729] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0f73f325af46babdf304fea3a0f6da, State: Initialized, Role: FOLLOWER
I20260926 04:06:54.267223 18830 raft_consensus.cc:729] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c3752c1d514cc7972ff931f00955c7, State: Initialized, Role: FOLLOWER
I20260926 04:06:54.267381 18830 consensus_queue.cc:260] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.267405 18829 consensus_queue.cc:260] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.267676 18830 ts_tablet_manager.cc:1428] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:54.267685 18829 ts_tablet_manager.cc:1428] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 04:06:54.267719 18553 heartbeater.cc:503] Master 127.0.200.190:35797 was elected leader, sending a full tablet report...
I20260926 04:06:54.267843 18815 heartbeater.cc:503] Master 127.0.200.190:35797 was elected leader, sending a full tablet report...
I20260926 04:06:54.267836 18831 raft_consensus.cc:348] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.267982 18831 raft_consensus.cc:374] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:54.268013 18831 raft_consensus.cc:729] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629833be2d714af389a73518d03c8960, State: Initialized, Role: FOLLOWER
I20260926 04:06:54.268163 18831 consensus_queue.cc:260] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.268427 18831 ts_tablet_manager.cc:1428] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260926 04:06:54.268478 18684 heartbeater.cc:503] Master 127.0.200.190:35797 was elected leader, sending a full tablet report...
W20260926 04:06:54.339218 18685 tablet.cc:2369] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:54.339497 18835 raft_consensus.cc:482] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:54.339632 18835 raft_consensus.cc:504] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.340015 18835 leader_election.cc:288] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 629833be2d714af389a73518d03c8960 (127.0.200.130:41423), 2c0f73f325af46babdf304fea3a0f6da (127.0.200.129:40515)
I20260926 04:06:54.344172 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.340374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:46023 (local address 127.0.200.129:40515)
0926 04:06:54.340422 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.340425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.340435 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:54.340469 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.340469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.340481 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.340563 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:54.340787 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.341539 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343180 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.343748 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343833 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.343842 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.344057 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.344099 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:06:54.344172 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.340382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:57363 (local address 127.0.200.130:41423)
0926 04:06:54.340440 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.340442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.340500 (+ 58us) server_negotiation.cc:407] Connection header received
0926 04:06:54.340594 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.340595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.340606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.340652 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:54.340796 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.341539 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343170 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.343756 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343833 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.343842 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.344057 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.344098 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:06:54.344208 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.340248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:40515 (local address 127.0.200.131:46023)
0926 04:06:54.340308 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.340321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.340382 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.340588 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.340589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.340591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:54.340662 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.340663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.341649 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.341650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.343065 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.343069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.343872 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.343959 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.343965 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.344034 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.344090 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":109}
I20260926 04:06:54.344219 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.340277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:41423 (local address 127.0.200.131:57363)
0926 04:06:54.340449 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.340459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.340513 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:54.340663 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:54.340663 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:54.340664 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:54.340722 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.340723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.341649 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.341650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.343065 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:54.343069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.343872 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.343874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:54.343971 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.343974 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:54.344034 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 04:06:54.344088 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":84,"thread_start_us":92,"threads_started":1}
I20260926 04:06:54.344731 18507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412e686a1201409e96183cc2dd264b0e" candidate_uuid: "00c3752c1d514cc7972ff931f00955c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0f73f325af46babdf304fea3a0f6da" is_pre_election: true
I20260926 04:06:54.344910 18507 raft_consensus.cc:2454] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c3752c1d514cc7972ff931f00955c7 in term 0.
I20260926 04:06:54.345148 18638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412e686a1201409e96183cc2dd264b0e" candidate_uuid: "00c3752c1d514cc7972ff931f00955c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629833be2d714af389a73518d03c8960" is_pre_election: true
I20260926 04:06:54.345216 18703 leader_election.cc:302] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c3752c1d514cc7972ff931f00955c7, 2c0f73f325af46babdf304fea3a0f6da; no voters:
I20260926 04:06:54.345310 18638 raft_consensus.cc:2454] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00c3752c1d514cc7972ff931f00955c7 in term 0.
I20260926 04:06:54.345378 18835 raft_consensus.cc:2781] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:54.345481 18835 raft_consensus.cc:482] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:54.345520 18835 raft_consensus.cc:3037] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:54.346288 18835 raft_consensus.cc:504] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.346454 18835 leader_election.cc:288] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [CANDIDATE]: Term 1 election: Requested vote from peers 629833be2d714af389a73518d03c8960 (127.0.200.130:41423), 2c0f73f325af46babdf304fea3a0f6da (127.0.200.129:40515)
I20260926 04:06:54.346645 18638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412e686a1201409e96183cc2dd264b0e" candidate_uuid: "00c3752c1d514cc7972ff931f00955c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "629833be2d714af389a73518d03c8960"
I20260926 04:06:54.346758 18638 raft_consensus.cc:3037] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:54.347502 18638 raft_consensus.cc:2454] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c3752c1d514cc7972ff931f00955c7 in term 1.
I20260926 04:06:54.347620 18507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412e686a1201409e96183cc2dd264b0e" candidate_uuid: "00c3752c1d514cc7972ff931f00955c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0f73f325af46babdf304fea3a0f6da"
I20260926 04:06:54.347707 18704 leader_election.cc:302] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c3752c1d514cc7972ff931f00955c7, 629833be2d714af389a73518d03c8960; no voters:
I20260926 04:06:54.347793 18835 raft_consensus.cc:2781] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:54.347728 18507 raft_consensus.cc:3037] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:54.347981 18835 raft_consensus.cc:686] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 1 LEADER]: Becoming Leader. State: Replica: 00c3752c1d514cc7972ff931f00955c7, State: Running, Role: LEADER
I20260926 04:06:54.348138 18835 consensus_queue.cc:237] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } }
I20260926 04:06:54.348686 18507 raft_consensus.cc:2454] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00c3752c1d514cc7972ff931f00955c7 in term 1.
I20260926 04:06:54.349033 18364 catalog_manager.cc:5495] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00c3752c1d514cc7972ff931f00955c7 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "00c3752c1d514cc7972ff931f00955c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c3752c1d514cc7972ff931f00955c7" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 } health_report { overall_health: UNKNOWN } } }
I20260926 04:06:54.375897 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:54.373342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.0.200.131:46761)
0926 04:06:54.373417 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:54.373419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:54.373429 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:54.373502 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:54.373503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:54.373521 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:54.373607 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:54.373748 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.374225 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.375055 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:54.375255 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:54.375339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:54.375512 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:54.375621 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:54.375736 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:54.375754 (+ 18us) server_negotiation.cc:299] Negotiation successful
0926 04:06:54.375773 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 04:06:54.377858 18507 raft_consensus.cc:1261] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 1 FOLLOWER]: Refusing update from remote peer 00c3752c1d514cc7972ff931f00955c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:54.377858 18638 raft_consensus.cc:1261] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 1 FOLLOWER]: Refusing update from remote peer 00c3752c1d514cc7972ff931f00955c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:54.378279 18839 consensus_queue.cc:1035] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0f73f325af46babdf304fea3a0f6da" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:54.378448 18835 consensus_queue.cc:1035] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "629833be2d714af389a73518d03c8960" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 41423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:54.381536 18843 mvcc.cc:205] Tried to move back new op lower bound from 7333460436489039872 to 7333460436371066880. Current Snapshot: MvccSnapshot[applied={T|T < 7333460436489039872}]
I20260926 04:06:54.383798 18365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:54.383889 18365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:54.384780 18365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a70dfdccfa8a4190baf323b1788af120: Sending DeleteTablet for 3 replicas of tablet 412e686a1201409e96183cc2dd264b0e
I20260926 04:06:54.385157 18618 tablet_service.cc:1514] Processing DeleteTablet for tablet 412e686a1201409e96183cc2dd264b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:54 UTC) from {username='slave'} at 127.0.0.1:38888
I20260926 04:06:54.385140 18487 tablet_service.cc:1514] Processing DeleteTablet for tablet 412e686a1201409e96183cc2dd264b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:54 UTC) from {username='slave'} at 127.0.0.1:36654
I20260926 04:06:54.385375 18749 tablet_service.cc:1514] Processing DeleteTablet for tablet 412e686a1201409e96183cc2dd264b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:54 UTC) from {username='slave'} at 127.0.0.1:50678
I20260926 04:06:54.385403 18853 tablet_replica.cc:330] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: stopping tablet replica
I20260926 04:06:54.385514 18853 raft_consensus.cc:2229] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:54.385584 18854 tablet_replica.cc:330] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: stopping tablet replica
I20260926 04:06:54.385613 18853 raft_consensus.cc:2258] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:54.385710 18854 raft_consensus.cc:2229] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:54.385824 18854 raft_consensus.cc:2258] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:54.386112 18853 ts_tablet_manager.cc:1905] T 412e686a1201409e96183cc2dd264b0e P 2c0f73f325af46babdf304fea3a0f6da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:54.386144 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18425
I20260926 04:06:54.386235 18854 ts_tablet_manager.cc:1905] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:54.387496 18852 tablet_replica.cc:330] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: stopping tablet replica
I20260926 04:06:54.387600 18852 raft_consensus.cc:2229] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:54.387702 18852 raft_consensus.cc:2258] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:54.388142 18852 ts_tablet_manager.cc:1905] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:54.388630 18854 ts_tablet_manager.cc:1918] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:54.388720 18854 log.cc:1196] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/412e686a1201409e96183cc2dd264b0e
I20260926 04:06:54.388973 18854 ts_tablet_manager.cc:1939] T 412e686a1201409e96183cc2dd264b0e P 00c3752c1d514cc7972ff931f00955c7: Deleting consensus metadata
I20260926 04:06:54.389572 18352 catalog_manager.cc:4841] TS 00c3752c1d514cc7972ff931f00955c7 (127.0.200.131:46761): tablet 412e686a1201409e96183cc2dd264b0e (table test-table [id=dd285a7352184146a7eb6f0cd5277272]) successfully deleted
I20260926 04:06:54.389794 18852 ts_tablet_manager.cc:1918] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:54.389856 18852 log.cc:1196] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/412e686a1201409e96183cc2dd264b0e
I20260926 04:06:54.390103 18852 ts_tablet_manager.cc:1939] T 412e686a1201409e96183cc2dd264b0e P 629833be2d714af389a73518d03c8960: Deleting consensus metadata
I20260926 04:06:54.390576 18351 catalog_manager.cc:4841] TS 629833be2d714af389a73518d03c8960 (127.0.200.130:41423): tablet 412e686a1201409e96183cc2dd264b0e (table test-table [id=dd285a7352184146a7eb6f0cd5277272]) successfully deleted
W20260926 04:06:54.391817 18350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:40515 (error 108)
W20260926 04:06:54.392354 18824 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:06:54.392194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:40515 (local address 127.0.0.1:36658)
0926 04:06:54.392253 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:54.392265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:54.392305 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:40515: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 04:06:54.392460 18350 catalog_manager.cc:4584] TS 2c0f73f325af46babdf304fea3a0f6da (127.0.200.129:40515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 412e686a1201409e96183cc2dd264b0e: Network error: Client connection negotiation failed: client connection to 127.0.200.129:40515: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260926 04:06:54.392757 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18556
I20260926 04:06:54.397707 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18687
I20260926 04:06:54.401997 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18334
2026-09-26T04:06:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (790 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:06:54 dist-test-slave-j8c1 krb5kdc[18858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:06:54 dist-test-slave-j8c1 krb5kdc[18858](info): set up 2 sockets
Sep 26 04:06:54 dist-test-slave-j8c1 krb5kdc[18858](info): commencing operation
krb5kdc: starting...
W20260926 04:06:55.454507 802 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:06:55 dist-test-slave-j8c1 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:06:55.593941 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43111
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:55.676013 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:55.676229 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:55.676254 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:55.677444 18871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:55.677495 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:55.677508 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:55.677521 18871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:55.677533 18871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:55.679116 18871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:55.679392 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:55.679659 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:55.684634 18879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:55.684650 18876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:55.684726 18877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:55.684872 18871 server_base.cc:1034] running on GCE node
I20260926 04:06:55.685135 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:55.685467 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:55.686689 18871 hybrid_clock.cc:648] HybridClock initialized: now 1790395615686679 us; error 30 us; skew 500 ppm
Sep 26 04:06:55 dist-test-slave-j8c1 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395615, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:55.689352 18871 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:06:55.689865 18871 webserver.cc:460] Webserver started at http://127.0.200.190:37313/ using document root <none> and password file <none>
I20260926 04:06:55.690122 18871 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:55.690196 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:55.690343 18871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:55.691464 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "368d2aa1b12c475b979f97df277ead04"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "185f934a3fcc509dbe60e132dc1f343b"
server_key_iv: "5e702c29f1657f441e472779eeed280b"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.691866 18871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "368d2aa1b12c475b979f97df277ead04"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "185f934a3fcc509dbe60e132dc1f343b"
server_key_iv: "5e702c29f1657f441e472779eeed280b"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.693190 18871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:55.693974 18886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:55.694149 18871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:06:55.694252 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "368d2aa1b12c475b979f97df277ead04"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "185f934a3fcc509dbe60e132dc1f343b"
server_key_iv: "5e702c29f1657f441e472779eeed280b"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.694339 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:55.716325 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:55.718787 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:55.718964 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:55.723150 18871 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43111
I20260926 04:06:55.723186 18938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43111 every 8 connection(s)
I20260926 04:06:55.723542 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:55.724413 18939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:55.726734 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: Bootstrap starting.
I20260926 04:06:55.727300 18939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:55.727594 18939 log.cc:824] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:55.728324 18939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: No bootstrap required, opened a new log
I20260926 04:06:55.729991 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18871
I20260926 04:06:55.730108 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:55.730020 18939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } }
I20260926 04:06:55.730225 18939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:55.730263 18939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 368d2aa1b12c475b979f97df277ead04, State: Initialized, Role: FOLLOWER
I20260926 04:06:55.730317 802 external_mini_cluster.cc:1428] Setting key 3275b96015e67ab7944acb18f6351e11
I20260926 04:06:55.730419 18939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } }
I20260926 04:06:55.730509 18939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:55.730575 18939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:55.730631 18939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:55.731609 18939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } }
I20260926 04:06:55.731705 18939 leader_election.cc:302] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 368d2aa1b12c475b979f97df277ead04; no voters:
I20260926 04:06:55.731882 18939 leader_election.cc:288] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:55.731950 18944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:55.732100 18944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [term 1 LEADER]: Becoming Leader. State: Replica: 368d2aa1b12c475b979f97df277ead04, State: Running, Role: LEADER
I20260926 04:06:55.732167 18939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:55.732192 18944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } }
I20260926 04:06:55.732513 18944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 368d2aa1b12c475b979f97df277ead04. Latest consensus state: current_term: 1 leader_uuid: "368d2aa1b12c475b979f97df277ead04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } } }
I20260926 04:06:55.732587 18944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:55.732538 18946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "368d2aa1b12c475b979f97df277ead04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "368d2aa1b12c475b979f97df277ead04" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43111 } } }
I20260926 04:06:55.732740 18946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04 [sys.catalog]: This master's current role is: LEADER
W20260926 04:06:55.733985 18959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:06:55.734086 18959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:06:55.734382 18950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:55.734580 18950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:55.737159 18950 catalog_manager.cc:1347] Generated new cluster ID: a1f879f5d11b458d93e0372f9af74396
I20260926 04:06:55.737216 18950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:06:55 dist-test-slave-j8c1 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:55.741505 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:55.730885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.0.200.190:43111)
0926 04:06:55.731065 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:55.731076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:55.731113 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:06:55.731230 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:55.731236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:55.731278 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:55.731415 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:55.732274 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.733667 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.734776 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.735308 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.735415 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:55.738117 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:55.738121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:55.738121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:55.738141 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:55.739908 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:55.740306 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:55.740309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:55.740309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:55.740417 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:55.740636 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:55.740637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:55.740637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:55.740902 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:55.741102 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:55.741181 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 04:06:55.741327 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260926 04:06:55.754698 18950 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:55.755456 18950 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:55.758432 18950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: Generated new TSK 0
I20260926 04:06:55.758704 18950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:06:55.794516 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:55.877199 18965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:55.877473 18965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:55.877512 18965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:55.878748 18965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:55.878831 18965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:55.878880 18965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:55.880442 18965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:55.880754 18965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:55.881013 18965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:55.886507 18970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:55.886523 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:55.886507 18973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:55.886528 18965 server_base.cc:1034] running on GCE node
I20260926 04:06:55.887037 18965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:55.887334 18965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:55.888506 18965 hybrid_clock.cc:648] HybridClock initialized: now 1790395615888499 us; error 74 us; skew 500 ppm
Sep 26 04:06:55 dist-test-slave-j8c1 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395615, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:55.891327 18965 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:06:55.891803 18965 webserver.cc:460] Webserver started at http://127.0.200.129:34067/ using document root <none> and password file <none>
I20260926 04:06:55.892050 18965 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:55.892130 18965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:55.892277 18965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:55.893328 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "019334487b824919ad74cc0c210f1fdb"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "204e0a06286b51329fb6a808b439935e"
server_key_iv: "1743d23975471a9db2870f770889d5f1"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.893712 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "019334487b824919ad74cc0c210f1fdb"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "204e0a06286b51329fb6a808b439935e"
server_key_iv: "1743d23975471a9db2870f770889d5f1"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.895190 18965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:06:55.895956 18980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:55.896137 18965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:06:55.896255 18965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "019334487b824919ad74cc0c210f1fdb"
format_stamp: "Formatted at 2026-09-26 04:06:55 on dist-test-slave-j8c1"
server_key: "204e0a06286b51329fb6a808b439935e"
server_key_iv: "1743d23975471a9db2870f770889d5f1"
server_key_version: "encryptionkey@0"
I20260926 04:06:55.896335 18965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:55.909606 18965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:55.911996 18965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:55.912202 18965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:55.912503 18965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:55.912859 18965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:55.912921 18965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:55.912959 18965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:55.912987 18965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:55.919113 18965 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:42577
I20260926 04:06:55.919205 19093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:42577 every 8 connection(s)
I20260926 04:06:55.919554 18965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:55.920171 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 18965
I20260926 04:06:55.920285 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:55.920516 802 external_mini_cluster.cc:1428] Setting key 0a64202c02417b18b59c82229e13b974
Sep 26 04:06:55 dist-test-slave-j8c1 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395615, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
I20260926 04:06:55.931053 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:55.920474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:50659 (local address 127.0.200.190:43111)
0926 04:06:55.920540 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:55.920542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:55.920812 (+ 270us) server_negotiation.cc:407] Connection header received
0926 04:06:55.921871 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:55.921872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:55.921889 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:55.921947 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:55.922777 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.923304 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.924349 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.924585 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.924672 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:55.927382 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:55.927383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:55.927384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:55.927391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:55.929072 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:55.929546 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:55.929547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:55.929548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:55.929608 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:55.930142 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:55.930144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:55.930144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:55.930317 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:55.930382 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:55.930833 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 04:06:55.930969 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 04:06:55.931137 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:55.920384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43111 (local address 127.0.200.129:50659)
0926 04:06:55.920677 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:55.920725 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:55.921743 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:55.922023 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:55.922026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:55.922296 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:55.922673 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:55.922679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.923415 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.923416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:55.924240 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:55.924242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:55.924694 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:55.924695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:55.924822 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:55.925524 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:55.925533 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:55.927258 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:55.929161 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:55.929170 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:55.929172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:55.929406 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:55.929766 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:55.929767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:55.929767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:55.929885 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:55.930402 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:55.930408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:55.930613 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 04:06:55.930736 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 04:06:55.930900 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260926 04:06:55.931854 19094 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43111
I20260926 04:06:55.931957 19094 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:55.932197 19094 heartbeater.cc:511] Master 127.0.200.190:43111 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
I20260926 04:06:55.932888 18903 ts_manager.cc:194] Registered new tserver with Master: 019334487b824919ad74cc0c210f1fdb (127.0.200.129:42577)
I20260926 04:06:55.933733 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:50659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:06:55.951579 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:56.034137 19099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:56.034356 19099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:56.034381 19099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:56.035583 19099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:56.035625 19099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:56.035652 19099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:56.037214 19099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:56.037416 19099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:56.037595 19099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:56.042917 19107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.042917 19104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.042949 19105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.043205 19099 server_base.cc:1034] running on GCE node
I20260926 04:06:56.043426 19099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:56.043743 19099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:56.044937 19099 hybrid_clock.cc:648] HybridClock initialized: now 1790395616044929 us; error 58 us; skew 500 ppm
Sep 26 04:06:56 dist-test-slave-j8c1 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:56.047910 19099 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:06:56.048488 19099 webserver.cc:460] Webserver started at http://127.0.200.130:40051/ using document root <none> and password file <none>
I20260926 04:06:56.048766 19099 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:56.048846 19099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:56.049015 19099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:56.050113 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "ce0a02a4f5fd4017af080a4ecb66ace4"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "0bf2470d3419a96479fdd88f492cbaa1"
server_key_iv: "e59f8890d1111df354753b81fc8583a2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.050514 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "ce0a02a4f5fd4017af080a4ecb66ace4"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "0bf2470d3419a96479fdd88f492cbaa1"
server_key_iv: "e59f8890d1111df354753b81fc8583a2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.052081 19099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:56.053050 19114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.053336 19099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:56.053527 19099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "ce0a02a4f5fd4017af080a4ecb66ace4"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "0bf2470d3419a96479fdd88f492cbaa1"
server_key_iv: "e59f8890d1111df354753b81fc8583a2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.053628 19099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:56.064615 19099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:56.067194 19099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:56.067402 19099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:56.067667 19099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:56.068017 19099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:56.068078 19099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.068118 19099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:56.068148 19099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.074985 19099 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:35361
I20260926 04:06:56.075047 19227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:35361 every 8 connection(s)
I20260926 04:06:56.075395 19099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:56.076628 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19099
I20260926 04:06:56.076752 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:56.076941 802 external_mini_cluster.cc:1428] Setting key 21d86d271e33834e53d7f2a56306908b
Sep 26 04:06:56 dist-test-slave-j8c1 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
I20260926 04:06:56.087117 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.076241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:49163 (local address 127.0.200.190:43111)
0926 04:06:56.076288 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.076290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.076406 (+ 116us) server_negotiation.cc:407] Connection header received
0926 04:06:56.077525 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.077526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.077539 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.077596 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:56.078283 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.078913 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.080000 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.080306 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.080390 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.083362 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:56.083363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:56.083364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:56.083375 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:56.085316 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.085848 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.085850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.085850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.085936 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.086230 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.086231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.086232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.086405 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:56.086488 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.086902 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.087007 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:06:56.087343 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.076102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43111 (local address 127.0.200.130:49163)
0926 04:06:56.076283 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.076328 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.077408 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.077676 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.077681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.077975 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:56.078176 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.078180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.079032 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.079033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.079890 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.079892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.080453 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.080453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.080581 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.081380 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:56.081395 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:56.083222 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:56.085460 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:56.085469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:56.085471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:56.085747 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:56.086025 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:56.086026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:56.086026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:56.086142 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:56.086492 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:56.086498 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:56.086725 (+ 227us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.086878 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.087086 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260926 04:06:56.087965 19228 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43111
I20260926 04:06:56.088084 19228 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:56.088294 19228 heartbeater.cc:511] Master 127.0.200.190:43111 requested a full tablet report, sending...
I20260926 04:06:56.088773 18903 ts_manager.cc:194] Registered new tserver with Master: ce0a02a4f5fd4017af080a4ecb66ace4 (127.0.200.130:35361)
I20260926 04:06:56.089203 18903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:49163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:06:56.106470 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:56.187525 19233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:56.187796 19233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:56.187845 19233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:56.189005 19233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:56.189083 19233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:56.189128 19233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:56.190634 19233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:56.190912 19233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:56.191150 19233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:56.196461 19239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.196475 19241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.196542 19233 server_base.cc:1034] running on GCE node
W20260926 04:06:56.196475 19238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.197006 19233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:56.197257 19233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:56.198443 19233 hybrid_clock.cc:648] HybridClock initialized: now 1790395616198427 us; error 48 us; skew 500 ppm
Sep 26 04:06:56 dist-test-slave-j8c1 krb5kdc[18858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:56.201305 19233 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:06:56.201735 19233 webserver.cc:460] Webserver started at http://127.0.200.131:35321/ using document root <none> and password file <none>
I20260926 04:06:56.201989 19233 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:56.202062 19233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:56.202214 19233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:56.203339 19233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "bb5a949a687843d4bcf90154c9209c98"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "6d2dd681a8f1cc519becd60c8e2ac65c"
server_key_iv: "192d3952aa01d16419805411290deaf2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.203716 19233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "bb5a949a687843d4bcf90154c9209c98"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "6d2dd681a8f1cc519becd60c8e2ac65c"
server_key_iv: "192d3952aa01d16419805411290deaf2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.205094 19233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:06:56.205828 19248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.206020 19233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.206120 19233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "bb5a949a687843d4bcf90154c9209c98"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "6d2dd681a8f1cc519becd60c8e2ac65c"
server_key_iv: "192d3952aa01d16419805411290deaf2"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.206215 19233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:56.221982 19233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:56.224373 19233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:56.224606 19233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:56.224854 19233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:56.225173 19233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:56.225232 19233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.225276 19233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:56.225302 19233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.231339 19233 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:34817
I20260926 04:06:56.231419 19361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:34817 every 8 connection(s)
I20260926 04:06:56.231755 19233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
Sep 26 04:06:56 dist-test-slave-j8c1 krb5kdc[18858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:56.241887 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19233
I20260926 04:06:56.242000 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:06:56.242105 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.232677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:33955 (local address 127.0.200.190:43111)
0926 04:06:56.232771 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.232774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.232958 (+ 184us) server_negotiation.cc:407] Connection header received
0926 04:06:56.234201 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.234202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.234216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.234256 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:56.234992 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.235495 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.236503 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.236718 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.236836 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.239121 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:56.239123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:56.239123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:56.239131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:56.240511 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.240913 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.240914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.240914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.240970 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.241196 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.241197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.241197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.241319 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:56.241443 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.241878 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.241969 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 04:06:56.242210 802 external_mini_cluster.cc:1428] Setting key 4707fcab82dbe67bb1c6fc26a400ec76
I20260926 04:06:56.242270 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.232592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43111 (local address 127.0.200.131:33955)
0926 04:06:56.232808 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.232872 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.234082 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.234340 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.234343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.234707 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:56.234896 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.234900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.235598 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.235599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.236390 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.236392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.236843 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.236845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.236967 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.237451 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:56.237459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:56.239003 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:56.240613 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:56.240619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:56.240622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:56.240834 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:56.241036 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:56.241037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:56.241037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:56.241129 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:56.241473 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:56.241481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:56.241667 (+ 186us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.241807 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.242018 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
I20260926 04:06:56.242959 19362 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43111
I20260926 04:06:56.243072 19362 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:56.243264 19362 heartbeater.cc:511] Master 127.0.200.190:43111 requested a full tablet report, sending...
I20260926 04:06:56.243660 18902 ts_manager.cc:194] Registered new tserver with Master: bb5a949a687843d4bcf90154c9209c98 (127.0.200.131:34817)
I20260926 04:06:56.244030 18902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:33955
I20260926 04:06:56.254619 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:56.262641 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.256081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.0.200.190:43111)
0926 04:06:56.256149 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.256151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.256249 (+ 98us) server_negotiation.cc:407] Connection header received
0926 04:06:56.256295 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.256295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.256305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.256354 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:56.256956 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.257453 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.258216 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.258437 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.258565 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.259957 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:56.259958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:56.259959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:56.259965 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:56.261262 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.261693 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.261694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.261694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.261749 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:56.261991 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:56.261991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:56.261992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:56.262125 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:56.262206 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.262408 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.262460 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:06:56.264506 18902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:56.266049 18902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:56.273726 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:34817 (local address 127.0.0.1:39580)
0926 04:06:56.269581 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.269592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.269641 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.270069 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.270070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.270072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:56.270113 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.270115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271126 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.271127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.272476 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.272478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273330 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.273428 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273445 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.273548 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.273618 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":115,"threads_started":1}
I20260926 04:06:56.273828 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.0.200.130:35361)
0926 04:06:56.269524 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.269530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.269701 (+ 171us) server_negotiation.cc:407] Connection header received
0926 04:06:56.269745 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.269747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.269763 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.269831 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:56.270090 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271422 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.272688 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273435 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273518 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273577 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.273688 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.273741 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
I20260926 04:06:56.273890 19374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:42577 (local address 127.0.0.1:53136)
0926 04:06:56.269481 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.269528 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.269798 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.270068 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.270068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.270070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:56.270129 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.270131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271606 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.271607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.272693 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.272696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273621 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.273690 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273697 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.273746 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.273785 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"spinlock_wait_cycles":768,"thread_start_us":106,"threads_started":1}
I20260926 04:06:56.274139 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.0.200.129:42577)
0926 04:06:56.269780 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.269784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.269795 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:56.269877 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.269879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.269894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.269969 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:56.270219 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271492 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.272788 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273484 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273542 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273588 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.274048 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.274082 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":85,"threads_started":1}
I20260926 04:06:56.274411 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35361 (local address 127.0.0.1:38218)
0926 04:06:56.269501 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.269533 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.269606 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.269937 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.269940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.269943 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:56.270024 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.270026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271560 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.271561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.272593 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.272595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273530 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.273605 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273609 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.273672 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.273713 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260926 04:06:56.274493 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.269438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.0.200.131:34817)
0926 04:06:56.269635 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.269639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.269651 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:06:56.269723 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.269725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.269742 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.269980 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:56.270199 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.271002 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.272577 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.273209 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.273327 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.273397 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.274317 (+ 920us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.274388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260926 04:06:56.274994 19028 tablet_service.cc:1467] Processing CreateTablet for tablet da9e147673b045e0bb63227bb60fe9f1 (DEFAULT_TABLE table=test-table [id=cef31b88e733436a9028e3d27695aaba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:56.275185 19296 tablet_service.cc:1467] Processing CreateTablet for tablet da9e147673b045e0bb63227bb60fe9f1 (DEFAULT_TABLE table=test-table [id=cef31b88e733436a9028e3d27695aaba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:56.275239 19162 tablet_service.cc:1467] Processing CreateTablet for tablet da9e147673b045e0bb63227bb60fe9f1 (DEFAULT_TABLE table=test-table [id=cef31b88e733436a9028e3d27695aaba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:56.275393 19028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9e147673b045e0bb63227bb60fe9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:56.275480 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9e147673b045e0bb63227bb60fe9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:56.275622 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da9e147673b045e0bb63227bb60fe9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:56.278489 19379 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Bootstrap starting.
I20260926 04:06:56.279016 19381 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Bootstrap starting.
I20260926 04:06:56.279313 19379 tablet_bootstrap.cc:654] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:56.279580 19381 tablet_bootstrap.cc:654] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:56.279914 19379 log.cc:824] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:56.280156 19381 log.cc:824] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:56.280301 19380 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Bootstrap starting.
I20260926 04:06:56.280694 19379 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: No bootstrap required, opened a new log
I20260926 04:06:56.280788 19379 ts_tablet_manager.cc:1397] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:56.281011 19380 tablet_bootstrap.cc:654] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:56.281045 19381 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: No bootstrap required, opened a new log
I20260926 04:06:56.281101 19381 ts_tablet_manager.cc:1397] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:06:56.281649 19380 log.cc:824] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:56.282469 19380 tablet_bootstrap.cc:492] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: No bootstrap required, opened a new log
I20260926 04:06:56.282537 19380 ts_tablet_manager.cc:1397] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:56.282939 19379 raft_consensus.cc:348] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.283126 19379 raft_consensus.cc:374] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:56.283164 19379 raft_consensus.cc:729] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0a02a4f5fd4017af080a4ecb66ace4, State: Initialized, Role: FOLLOWER
I20260926 04:06:56.283214 19381 raft_consensus.cc:348] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.283286 19379 consensus_queue.cc:260] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.283375 19381 raft_consensus.cc:374] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:56.283412 19381 raft_consensus.cc:729] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019334487b824919ad74cc0c210f1fdb, State: Initialized, Role: FOLLOWER
I20260926 04:06:56.283529 19381 consensus_queue.cc:260] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.283617 19379 ts_tablet_manager.cc:1428] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:56.283735 19228 heartbeater.cc:503] Master 127.0.200.190:43111 was elected leader, sending a full tablet report...
I20260926 04:06:56.283799 19381 ts_tablet_manager.cc:1428] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:56.283870 19094 heartbeater.cc:503] Master 127.0.200.190:43111 was elected leader, sending a full tablet report...
I20260926 04:06:56.284766 19380 raft_consensus.cc:348] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.284977 19380 raft_consensus.cc:374] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:56.285035 19380 raft_consensus.cc:729] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5a949a687843d4bcf90154c9209c98, State: Initialized, Role: FOLLOWER
I20260926 04:06:56.285148 19380 consensus_queue.cc:260] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.285458 19380 ts_tablet_manager.cc:1428] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 04:06:56.285655 19362 heartbeater.cc:503] Master 127.0.200.190:43111 was elected leader, sending a full tablet report...
W20260926 04:06:56.325974 19229 tablet.cc:2369] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:06:56.420353 19095 tablet.cc:2369] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:06:56.482401 19363 tablet.cc:2369] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:56.497274 19385 raft_consensus.cc:482] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:56.497457 19385 raft_consensus.cc:504] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.497934 19385 leader_election.cc:288] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019334487b824919ad74cc0c210f1fdb (127.0.200.129:42577), bb5a949a687843d4bcf90154c9209c98 (127.0.200.131:34817)
I20260926 04:06:56.501124 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.498199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:56043 (local address 127.0.200.129:42577)
0926 04:06:56.498285 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.498288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.498300 (+ 12us) server_negotiation.cc:407] Connection header received
0926 04:06:56.498360 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.498361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.498371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.498451 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:56.498635 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.499262 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.500313 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.500801 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.500855 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.500862 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.501045 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.501074 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 04:06:56.501139 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.498146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:42577 (local address 127.0.200.130:56043)
0926 04:06:56.498221 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.498234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.498292 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.498454 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.498455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.498457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:56.498533 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.498535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.499345 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.499346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.500218 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.500221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.500886 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.500887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.500963 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.500968 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.501029 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.501063 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 04:06:56.501513 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9e147673b045e0bb63227bb60fe9f1" candidate_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019334487b824919ad74cc0c210f1fdb" is_pre_election: true
I20260926 04:06:56.501570 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.498148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:34817 (local address 127.0.200.130:53823)
0926 04:06:56.498375 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.498391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.498479 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.498629 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.498630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.498632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:56.498716 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.498718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.499650 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.499651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.500561 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.500563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.501338 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.501339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.501408 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.501412 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.501462 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.501497 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":67,"thread_start_us":104,"threads_started":1}
I20260926 04:06:56.501564 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.498222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:53823 (local address 127.0.200.131:34817)
0926 04:06:56.498282 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.498284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.498463 (+ 179us) server_negotiation.cc:407] Connection header received
0926 04:06:56.498536 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.498537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.498576 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.498628 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:56.498819 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.499572 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.500647 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.501154 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.501276 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.501286 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.501468 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.501505 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:06:56.501710 19048 raft_consensus.cc:2454] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0a02a4f5fd4017af080a4ecb66ace4 in term 0.
I20260926 04:06:56.501964 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9e147673b045e0bb63227bb60fe9f1" candidate_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb5a949a687843d4bcf90154c9209c98" is_pre_election: true
I20260926 04:06:56.502000 19117 leader_election.cc:302] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019334487b824919ad74cc0c210f1fdb, ce0a02a4f5fd4017af080a4ecb66ace4; no voters:
I20260926 04:06:56.502131 19316 raft_consensus.cc:2454] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0a02a4f5fd4017af080a4ecb66ace4 in term 0.
I20260926 04:06:56.502128 19385 raft_consensus.cc:2781] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:56.502183 19385 raft_consensus.cc:482] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:56.502239 19385 raft_consensus.cc:3037] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:56.502943 19385 raft_consensus.cc:504] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.503135 19385 leader_election.cc:288] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [CANDIDATE]: Term 1 election: Requested vote from peers 019334487b824919ad74cc0c210f1fdb (127.0.200.129:42577), bb5a949a687843d4bcf90154c9209c98 (127.0.200.131:34817)
I20260926 04:06:56.503288 19316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9e147673b045e0bb63227bb60fe9f1" candidate_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb5a949a687843d4bcf90154c9209c98"
I20260926 04:06:56.503351 19316 raft_consensus.cc:3037] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:56.503285 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da9e147673b045e0bb63227bb60fe9f1" candidate_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019334487b824919ad74cc0c210f1fdb"
I20260926 04:06:56.503419 19048 raft_consensus.cc:3037] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:56.503985 19316 raft_consensus.cc:2454] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0a02a4f5fd4017af080a4ecb66ace4 in term 1.
I20260926 04:06:56.504146 19116 leader_election.cc:302] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb5a949a687843d4bcf90154c9209c98, ce0a02a4f5fd4017af080a4ecb66ace4; no voters:
I20260926 04:06:56.504256 19385 raft_consensus.cc:2781] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:56.504221 19048 raft_consensus.cc:2454] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0a02a4f5fd4017af080a4ecb66ace4 in term 1.
I20260926 04:06:56.504408 19385 raft_consensus.cc:686] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 1 LEADER]: Becoming Leader. State: Replica: ce0a02a4f5fd4017af080a4ecb66ace4, State: Running, Role: LEADER
I20260926 04:06:56.504529 19385 consensus_queue.cc:237] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } }
I20260926 04:06:56.505504 18903 catalog_manager.cc:5495] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0a02a4f5fd4017af080a4ecb66ace4 (127.0.200.130). New cstate: current_term: 1 leader_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce0a02a4f5fd4017af080a4ecb66ace4" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 } health_report { overall_health: UNKNOWN } } }
I20260926 04:06:56.560273 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.557723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.0.200.130:35361)
0926 04:06:56.557818 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.557820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.557829 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:06:56.557866 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.557867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.557885 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.557967 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:56.558107 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.558722 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.559525 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.559726 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.559788 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.559934 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:56.560025 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:56.560104 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.560165 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.560175 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 04:06:56.562227 19048 raft_consensus.cc:1261] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 1 FOLLOWER]: Refusing update from remote peer ce0a02a4f5fd4017af080a4ecb66ace4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:56.562345 19316 raft_consensus.cc:1261] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 1 FOLLOWER]: Refusing update from remote peer ce0a02a4f5fd4017af080a4ecb66ace4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:56.562678 19389 consensus_queue.cc:1035] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019334487b824919ad74cc0c210f1fdb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:56.562867 19385 consensus_queue.cc:1035] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb5a949a687843d4bcf90154c9209c98" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 34817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:56.565164 19392 mvcc.cc:205] Tried to move back new op lower bound from 7333460445435924480 to 7333460445203972096. Current Snapshot: MvccSnapshot[applied={T|T < 7333460445435924480}]
I20260926 04:06:56.566162 19394 mvcc.cc:205] Tried to move back new op lower bound from 7333460445435924480 to 7333460445203972096. Current Snapshot: MvccSnapshot[applied={T|T < 7333460445435924480 or (T in {7333460445435924480})}]
I20260926 04:06:56.567638 18902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:56.567732 18902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:56.568665 18902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 368d2aa1b12c475b979f97df277ead04: Sending DeleteTablet for 3 replicas of tablet da9e147673b045e0bb63227bb60fe9f1
I20260926 04:06:56.569063 19028 tablet_service.cc:1514] Processing DeleteTablet for tablet da9e147673b045e0bb63227bb60fe9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:56 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:53136
I20260926 04:06:56.569169 19162 tablet_service.cc:1514] Processing DeleteTablet for tablet da9e147673b045e0bb63227bb60fe9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:56 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:38218
I20260926 04:06:56.569350 19402 tablet_replica.cc:330] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb: stopping tablet replica
I20260926 04:06:56.569410 19296 tablet_service.cc:1514] Processing DeleteTablet for tablet da9e147673b045e0bb63227bb60fe9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:56 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:39580
I20260926 04:06:56.569463 19402 raft_consensus.cc:2229] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:56.569561 19403 tablet_replica.cc:330] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: stopping tablet replica
I20260926 04:06:56.569578 19402 raft_consensus.cc:2258] T da9e147673b045e0bb63227bb60fe9f1 P 019334487b824919ad74cc0c210f1fdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:56.569728 19403 raft_consensus.cc:2229] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:56.569845 19404 tablet_replica.cc:330] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: stopping tablet replica
I20260926 04:06:56.569869 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18965
I20260926 04:06:56.569950 19404 raft_consensus.cc:2229] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:56.570051 19404 raft_consensus.cc:2258] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:56.570502 19403 raft_consensus.cc:2258] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:56.570510 19404 ts_tablet_manager.cc:1905] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:56.571061 19403 ts_tablet_manager.cc:1905] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:56.572553 19404 ts_tablet_manager.cc:1918] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:56.572638 19404 log.cc:1196] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/da9e147673b045e0bb63227bb60fe9f1
I20260926 04:06:56.572956 19404 ts_tablet_manager.cc:1939] T da9e147673b045e0bb63227bb60fe9f1 P bb5a949a687843d4bcf90154c9209c98: Deleting consensus metadata
I20260926 04:06:56.573310 19403 ts_tablet_manager.cc:1918] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:56.573441 19403 log.cc:1196] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/da9e147673b045e0bb63227bb60fe9f1
I20260926 04:06:56.573575 18888 catalog_manager.cc:4841] TS bb5a949a687843d4bcf90154c9209c98 (127.0.200.131:34817): tablet da9e147673b045e0bb63227bb60fe9f1 (table test-table [id=cef31b88e733436a9028e3d27695aaba]) successfully deleted
I20260926 04:06:56.573838 19403 ts_tablet_manager.cc:1939] T da9e147673b045e0bb63227bb60fe9f1 P ce0a02a4f5fd4017af080a4ecb66ace4: Deleting consensus metadata
I20260926 04:06:56.574641 18888 catalog_manager.cc:4841] TS ce0a02a4f5fd4017af080a4ecb66ace4 (127.0.200.130:35361): tablet da9e147673b045e0bb63227bb60fe9f1 (table test-table [id=cef31b88e733436a9028e3d27695aaba]) successfully deleted
W20260926 04:06:56.578850 18889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:42577 (error 108)
I20260926 04:06:56.579063 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19099
I20260926 04:06:56.579967 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.579809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:42577 (local address 127.0.0.1:34270)
0926 04:06:56.579879 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.579921 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:42577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 04:06:56.580367 18889 catalog_manager.cc:4584] TS 019334487b824919ad74cc0c210f1fdb (127.0.200.129:42577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da9e147673b045e0bb63227bb60fe9f1: Network error: Client connection negotiation failed: client connection to 127.0.200.129:42577: connect: Connection refused (error 111)
I20260926 04:06:56.583714 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19233
I20260926 04:06:56.588462 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 18871
2026-09-26T04:06:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2188 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T04:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:56Z Disabled control of system clock
I20260926 04:06:56.617676 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36527
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:56.701051 19409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:56.701293 19409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:56.701315 19409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:56.702600 19409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:56.702643 19409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:56.702656 19409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:56.702669 19409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:56.702684 19409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:56.704062 19409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:56.704272 19409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:56.704510 19409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:56.709573 19414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.709554 19417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.709744 19409 server_base.cc:1034] running on GCE node
W20260926 04:06:56.709645 19415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.710119 19409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:56.710500 19409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:56.711705 19409 hybrid_clock.cc:648] HybridClock initialized: now 1790395616711682 us; error 40 us; skew 500 ppm
I20260926 04:06:56.717571 19409 webserver.cc:460] Webserver started at http://127.0.200.190:46455/ using document root <none> and password file <none>
I20260926 04:06:56.717865 19409 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:56.717949 19409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:56.718146 19409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:56.719398 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "bebdf290399247ddb88c59ed2845f064"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "239bc6167b211000483c53e28eccb9d1"
server_key_iv: "b9b439c6da51e9f044e27f01a5731f52"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.719784 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "bebdf290399247ddb88c59ed2845f064"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "239bc6167b211000483c53e28eccb9d1"
server_key_iv: "b9b439c6da51e9f044e27f01a5731f52"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.721133 19409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:06:56.721971 19423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.722240 19409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:56.722349 19409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "bebdf290399247ddb88c59ed2845f064"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "239bc6167b211000483c53e28eccb9d1"
server_key_iv: "b9b439c6da51e9f044e27f01a5731f52"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.722425 19409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:56.730070 19409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:56.730423 19409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:56.730602 19409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:56.735081 19409 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:36527
I20260926 04:06:56.735113 19475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:36527 every 8 connection(s)
I20260926 04:06:56.735494 19409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:56.736503 19476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:56.738940 19476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: Bootstrap starting.
I20260926 04:06:56.739488 19476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:56.739737 19476 log.cc:824] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:56.740504 19476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: No bootstrap required, opened a new log
I20260926 04:06:56.742110 19476 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } }
I20260926 04:06:56.742264 19476 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:56.742286 19476 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebdf290399247ddb88c59ed2845f064, State: Initialized, Role: FOLLOWER
I20260926 04:06:56.742451 19476 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } }
I20260926 04:06:56.742537 19476 raft_consensus.cc:388] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:56.742614 19476 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:56.742630 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19409
I20260926 04:06:56.742667 19476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:56.742745 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:56.742973 802 external_mini_cluster.cc:1428] Setting key 09b1ec3c510b3a2a621679c8a4e693fb
I20260926 04:06:56.743426 19476 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } }
I20260926 04:06:56.743546 19476 leader_election.cc:302] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bebdf290399247ddb88c59ed2845f064; no voters:
I20260926 04:06:56.743711 19476 leader_election.cc:288] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:56.743773 19481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:56.743906 19481 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [term 1 LEADER]: Becoming Leader. State: Replica: bebdf290399247ddb88c59ed2845f064, State: Running, Role: LEADER
I20260926 04:06:56.744020 19476 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:56.743996 19481 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } }
I20260926 04:06:56.744518 19482 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebdf290399247ddb88c59ed2845f064. Latest consensus state: current_term: 1 leader_uuid: "bebdf290399247ddb88c59ed2845f064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } } }
I20260926 04:06:56.744623 19482 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:56.744607 19483 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bebdf290399247ddb88c59ed2845f064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebdf290399247ddb88c59ed2845f064" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36527 } } }
I20260926 04:06:56.745010 19483 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:56.745309 19490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:56.745908 19490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:56.748096 19490 catalog_manager.cc:1347] Generated new cluster ID: 43a4a0630f7a4b3f8d33f9278a11035c
I20260926 04:06:56.748148 19490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:06:56.749552 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.743537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.0.200.190:36527)
0926 04:06:56.743828 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.743840 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.743880 (+ 40us) server_negotiation.cc:407] Connection header received
0926 04:06:56.743999 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.744007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.744082 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.744500 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:56.744614 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.746186 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.747152 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.747753 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.747897 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.748660 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:56.748664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:56.748664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:56.748895 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:56.748941 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:56.748956 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:56.749031 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.749108 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.749307 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260926 04:06:56.754832 19490 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:56.755544 19490 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:56.765717 19490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: Generated new TSK 0
I20260926 04:06:56.765969 19490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:06:56.769860 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--builtin_ntp_servers=127.0.200.148:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:56.860244 19500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:56.860471 19500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:56.860496 19500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:56.861743 19500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:56.861793 19500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:56.861820 19500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:56.863339 19500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:56.863643 19500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:56.864009 19500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:56.869462 19508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.869447 19505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.869823 19500 server_base.cc:1034] running on GCE node
W20260926 04:06:56.869449 19506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.870466 19500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:56.870785 19500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:56.871968 19500 hybrid_clock.cc:648] HybridClock initialized: now 1790395616871910 us; error 79 us; skew 500 ppm
I20260926 04:06:56.873468 19500 webserver.cc:460] Webserver started at http://127.0.200.129:41835/ using document root <none> and password file <none>
I20260926 04:06:56.873778 19500 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:56.873831 19500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:56.874004 19500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:56.875217 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "69a78499161242fe80effcb646b98e97"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "daf74bd1ae5b850bf9cddfb9683dcacc"
server_key_iv: "a73c5f9057a9a767dc61e04fd4dcf250"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.875645 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "69a78499161242fe80effcb646b98e97"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "daf74bd1ae5b850bf9cddfb9683dcacc"
server_key_iv: "a73c5f9057a9a767dc61e04fd4dcf250"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.876986 19500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:06:56.877820 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.878046 19500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:56.878150 19500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "69a78499161242fe80effcb646b98e97"
format_stamp: "Formatted at 2026-09-26 04:06:56 on dist-test-slave-j8c1"
server_key: "daf74bd1ae5b850bf9cddfb9683dcacc"
server_key_iv: "a73c5f9057a9a767dc61e04fd4dcf250"
server_key_version: "encryptionkey@0"
I20260926 04:06:56.878252 19500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:56.891983 19500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:56.892328 19500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:56.892488 19500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:56.892731 19500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:56.893061 19500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:56.893118 19500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.893160 19500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:56.893183 19500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:56.898911 19500 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:41233
I20260926 04:06:56.899006 19627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:41233 every 8 connection(s)
I20260926 04:06:56.899354 19500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:56.904862 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.900249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:44607 (local address 127.0.200.190:36527)
0926 04:06:56.900319 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:56.900321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:56.900553 (+ 232us) server_negotiation.cc:407] Connection header received
0926 04:06:56.901046 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:56.901047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:56.901060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:56.901124 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:56.901844 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.902410 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.903495 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.903709 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.903775 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.904542 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:56.904543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:56.904543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:56.904547 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:56.904561 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:56.904565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:56.904622 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:56.904762 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 04:06:56.904797 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:06:56.904951 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:56.900230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36527 (local address 127.0.200.129:44607)
0926 04:06:56.900421 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:56.900457 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:56.900954 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:56.901176 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:56.901179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:56.901187 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:56.901715 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.901734 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.902525 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.902526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.903399 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:56.903401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:56.903796 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:56.903797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:56.903905 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:56.904408 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:56.904408 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:56.904435 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:56.904459 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:56.904637 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:56.904640 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:56.904643 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:56.904695 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 04:06:56.904791 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260926 04:06:56.905529 19628 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36527
I20260926 04:06:56.905776 19628 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:56.905957 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19500
I20260926 04:06:56.906051 19628 heartbeater.cc:511] Master 127.0.200.190:36527 requested a full tablet report, sending...
I20260926 04:06:56.906055 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:56.906309 802 external_mini_cluster.cc:1428] Setting key f0dd61fb8471af21d3e7f5934217e0e6
I20260926 04:06:56.906754 19440 ts_manager.cc:194] Registered new tserver with Master: 69a78499161242fe80effcb646b98e97 (127.0.200.129:41233)
I20260926 04:06:56.907433 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--builtin_ntp_servers=127.0.200.148:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 04:06:56.907603 19440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.129:44607
W20260926 04:06:56.990309 19631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:56.990620 19631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:56.990665 19631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:56.991978 19631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:56.992079 19631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:56.992130 19631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:56.993650 19631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:56.993966 19631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:56.994309 19631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:56.999764 19637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.999766 19639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:56.999773 19636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:56.999967 19631 server_base.cc:1034] running on GCE node
I20260926 04:06:57.000461 19631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:57.000718 19631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:57.001878 19631 hybrid_clock.cc:648] HybridClock initialized: now 1790395617001843 us; error 55 us; skew 500 ppm
I20260926 04:06:57.003345 19631 webserver.cc:460] Webserver started at http://127.0.200.130:36351/ using document root <none> and password file <none>
I20260926 04:06:57.003612 19631 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:57.003711 19631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:57.003868 19631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:57.005004 19631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "52a8440f88b34155995edea719cf910d"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "53b6e76297a238cdc53c1b0f71fa1911"
server_key_iv: "f697c9589054e18232429554f113eed8"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.005380 19631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "52a8440f88b34155995edea719cf910d"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "53b6e76297a238cdc53c1b0f71fa1911"
server_key_iv: "f697c9589054e18232429554f113eed8"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.006853 19631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:06:57.007737 19645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.008097 19631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:57.008320 19631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "52a8440f88b34155995edea719cf910d"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "53b6e76297a238cdc53c1b0f71fa1911"
server_key_iv: "f697c9589054e18232429554f113eed8"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.008447 19631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:57.018392 19631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:57.018787 19631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:57.018982 19631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:57.019317 19631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:57.019721 19631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:57.019788 19631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.019827 19631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:57.019850 19631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.025966 19631 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:34329
I20260926 04:06:57.026059 19758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:34329 every 8 connection(s)
I20260926 04:06:57.026441 19631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:57.032346 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.027250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:43497 (local address 127.0.200.190:36527)
0926 04:06:57.027328 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.027330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.027568 (+ 238us) server_negotiation.cc:407] Connection header received
0926 04:06:57.028070 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.028071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.028083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.028160 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:57.028829 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.029485 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.030864 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.031085 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.031147 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.032073 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:57.032074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:57.032075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:57.032079 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:57.032095 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:57.032099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:57.032170 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.032240 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.032268 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 04:06:57.032527 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.027234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36527 (local address 127.0.200.130:43497)
0926 04:06:57.027409 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.027451 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.027940 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.028213 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.028216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.028223 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:57.028724 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.028729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.029596 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.029597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.030781 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.030784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.031170 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.031171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.031280 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.031910 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:57.031911 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:57.031939 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:57.031962 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:57.032172 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:57.032174 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:57.032177 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.032224 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.032300 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260926 04:06:57.032732 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19631
I20260926 04:06:57.032810 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:57.033027 802 external_mini_cluster.cc:1428] Setting key 799ccd48bd8812e7ef1631255bd0333b
I20260926 04:06:57.033120 19759 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36527
I20260926 04:06:57.033236 19759 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:57.033432 19759 heartbeater.cc:511] Master 127.0.200.190:36527 requested a full tablet report, sending...
I20260926 04:06:57.033828 19440 ts_manager.cc:194] Registered new tserver with Master: 52a8440f88b34155995edea719cf910d (127.0.200.130:34329)
I20260926 04:06:57.033969 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--builtin_ntp_servers=127.0.200.148:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 04:06:57.034309 19440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.130:43497
W20260926 04:06:57.119798 19762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:57.120090 19762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:57.120129 19762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:57.121312 19762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:57.121387 19762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:57.121430 19762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:57.122815 19762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:57.123096 19762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:57.123399 19762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:57.128806 19768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:57.128808 19767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:57.128919 19770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:57.128835 19762 server_base.cc:1034] running on GCE node
I20260926 04:06:57.129335 19762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:57.129611 19762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:57.130776 19762 hybrid_clock.cc:648] HybridClock initialized: now 1790395617130752 us; error 39 us; skew 500 ppm
I20260926 04:06:57.132337 19762 webserver.cc:460] Webserver started at http://127.0.200.131:41193/ using document root <none> and password file <none>
I20260926 04:06:57.132658 19762 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:57.132714 19762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:57.132897 19762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:57.134109 19762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "b3a26fd9b6b84a92a3aa29adab1514f1"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "97a633b65ed24b0b5206031b8f9ef9eb"
server_key_iv: "9b454eb8e5df911942bcd6e0bdf0b71d"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.134536 19762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "b3a26fd9b6b84a92a3aa29adab1514f1"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "97a633b65ed24b0b5206031b8f9ef9eb"
server_key_iv: "9b454eb8e5df911942bcd6e0bdf0b71d"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.136126 19762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:57.136981 19776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.137271 19762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:57.137372 19762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "b3a26fd9b6b84a92a3aa29adab1514f1"
format_stamp: "Formatted at 2026-09-26 04:06:57 on dist-test-slave-j8c1"
server_key: "97a633b65ed24b0b5206031b8f9ef9eb"
server_key_iv: "9b454eb8e5df911942bcd6e0bdf0b71d"
server_key_version: "encryptionkey@0"
I20260926 04:06:57.137447 19762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:57.149430 19762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:57.149753 19762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:57.149865 19762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:57.150141 19762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:57.150482 19762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:57.150519 19762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.150563 19762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:57.150578 19762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:57.156446 19762 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:39831
I20260926 04:06:57.156493 19889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:39831 every 8 connection(s)
I20260926 04:06:57.156831 19762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:57.159335 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19762
I20260926 04:06:57.159478 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:06:57.159711 802 external_mini_cluster.cc:1428] Setting key bd8c199c74f86121782c2931a5b4d3c1
I20260926 04:06:57.162891 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.157658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:41867 (local address 127.0.200.190:36527)
0926 04:06:57.157753 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.157756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.157934 (+ 178us) server_negotiation.cc:407] Connection header received
0926 04:06:57.158524 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.158525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.158539 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.158607 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:57.159323 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.160059 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.161283 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.161510 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.161596 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.162513 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:57.162515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:57.162515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:57.162520 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:57.162534 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:57.162538 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:57.162615 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.162800 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.162829 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 04:06:57.163061 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.157538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36527 (local address 127.0.200.131:41867)
0926 04:06:57.157804 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.157851 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.158389 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.158674 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.158678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.158688 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:57.159224 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.159229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.160141 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.160142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.161193 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.161196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.161603 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.161604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.161743 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.162351 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:57.162351 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:57.162381 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:57.162405 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:57.162648 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:57.162654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:57.162658 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.162735 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.162827 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":109,"threads_started":1}
I20260926 04:06:57.163668 19890 heartbeater.cc:347] Connected to a master server at 127.0.200.190:36527
I20260926 04:06:57.163756 19890 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:57.163952 19890 heartbeater.cc:511] Master 127.0.200.190:36527 requested a full tablet report, sending...
I20260926 04:06:57.164430 19440 ts_manager.cc:194] Registered new tserver with Master: b3a26fd9b6b84a92a3aa29adab1514f1 (127.0.200.131:39831)
I20260926 04:06:57.164824 19440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.131:41867
I20260926 04:06:57.172175 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:57.176966 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.173684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39112 (local address 127.0.200.190:36527)
0926 04:06:57.173764 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.173766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.173799 (+ 33us) server_negotiation.cc:407] Connection header received
0926 04:06:57.173820 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.173821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.173833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.173876 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:57.174100 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.174655 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.175525 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.175751 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.175816 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.176561 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:57.176562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:57.176562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:57.176567 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:57.176581 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:57.176585 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:57.176680 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.176892 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.176904 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 04:06:57.178927 19440 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:57.180684 19440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:57.187320 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.0.200.130:34329)
0926 04:06:57.183626 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.183631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.183642 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:57.183683 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.183685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.183702 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.183785 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:57.183982 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.184788 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.186173 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.186890 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187005 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187101 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.187191 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.187229 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260926 04:06:57.187310 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:34329 (local address 127.0.0.1:58872)
0926 04:06:57.183369 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.183413 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.183490 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.183816 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.183819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.183821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:57.183893 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.183896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.184897 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.184898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.186074 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.186077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.187011 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.187087 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187099 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.187177 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.187230 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260926 04:06:57.187602 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:39831 (local address 127.0.0.1:41518)
0926 04:06:57.183445 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.183454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.183495 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.183884 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.183885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.183887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:57.183949 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.183950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.185414 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.185415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.186312 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.186315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.187350 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.187414 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187418 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.187478 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.187523 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":3,"thread_start_us":343,"threads_started":1}
I20260926 04:06:57.187821 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.0.200.131:39831)
0926 04:06:57.183671 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.183675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.183685 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:57.183721 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.183723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.183738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.183814 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:57.184030 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.185325 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.186404 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.187156 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187230 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187311 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.187679 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.187723 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":76,"threads_started":1}
I20260926 04:06:57.187899 19902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41233 (local address 127.0.0.1:42826)
0926 04:06:57.183570 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.183581 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.183638 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.183884 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.183885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.183888 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:57.183948 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.183950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.185405 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.185406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.186521 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.186524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.187513 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.187698 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187702 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.187774 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.187812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":45,"thread_start_us":88,"threads_started":1}
I20260926 04:06:57.188265 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.183286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.0.200.129:41233)
0926 04:06:57.183624 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.183629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.183640 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:57.183718 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.183720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.183736 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.183814 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:57.184029 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.185289 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.186623 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.187340 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.187401 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.187481 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.188122 (+ 641us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.188179 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":75,"threads_started":1}
I20260926 04:06:57.188666 19693 tablet_service.cc:1467] Processing CreateTablet for tablet 4309a0ccad874590b53b8d8a058ba916 (DEFAULT_TABLE table=test-table [id=f36f94e0bbb248b6ba9b74a3da44b83b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:57.188666 19824 tablet_service.cc:1467] Processing CreateTablet for tablet 4309a0ccad874590b53b8d8a058ba916 (DEFAULT_TABLE table=test-table [id=f36f94e0bbb248b6ba9b74a3da44b83b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:57.189044 19693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4309a0ccad874590b53b8d8a058ba916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:57.189076 19824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4309a0ccad874590b53b8d8a058ba916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:57.191416 19562 tablet_service.cc:1467] Processing CreateTablet for tablet 4309a0ccad874590b53b8d8a058ba916 (DEFAULT_TABLE table=test-table [id=f36f94e0bbb248b6ba9b74a3da44b83b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:57.191756 19562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4309a0ccad874590b53b8d8a058ba916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:57.192524 19905 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Bootstrap starting.
I20260926 04:06:57.192752 19904 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Bootstrap starting.
I20260926 04:06:57.193471 19904 tablet_bootstrap.cc:654] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:57.193826 19904 log.cc:824] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:57.194679 19905 tablet_bootstrap.cc:654] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:57.195024 19905 log.cc:824] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:57.195257 19907 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Bootstrap starting.
I20260926 04:06:57.196061 19907 tablet_bootstrap.cc:654] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:57.196395 19907 log.cc:824] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:57.197274 19905 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: No bootstrap required, opened a new log
I20260926 04:06:57.197383 19905 ts_tablet_manager.cc:1397] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260926 04:06:57.197444 19907 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: No bootstrap required, opened a new log
I20260926 04:06:57.197674 19907 ts_tablet_manager.cc:1397] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 04:06:57.199951 19904 tablet_bootstrap.cc:492] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: No bootstrap required, opened a new log
I20260926 04:06:57.200055 19904 ts_tablet_manager.cc:1397] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260926 04:06:57.200632 19905 raft_consensus.cc:348] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.200973 19905 raft_consensus.cc:374] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:57.201083 19905 raft_consensus.cc:729] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a8440f88b34155995edea719cf910d, State: Initialized, Role: FOLLOWER
I20260926 04:06:57.201292 19905 consensus_queue.cc:260] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.201675 19905 ts_tablet_manager.cc:1428] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260926 04:06:57.201848 19759 heartbeater.cc:503] Master 127.0.200.190:36527 was elected leader, sending a full tablet report...
I20260926 04:06:57.203086 19907 raft_consensus.cc:348] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.203289 19907 raft_consensus.cc:374] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:57.203331 19907 raft_consensus.cc:729] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a78499161242fe80effcb646b98e97, State: Initialized, Role: FOLLOWER
I20260926 04:06:57.203492 19907 consensus_queue.cc:260] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.203810 19907 ts_tablet_manager.cc:1428] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 04:06:57.204138 19628 heartbeater.cc:503] Master 127.0.200.190:36527 was elected leader, sending a full tablet report...
I20260926 04:06:57.205610 19904 raft_consensus.cc:348] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.205771 19904 raft_consensus.cc:374] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:57.205796 19904 raft_consensus.cc:729] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a26fd9b6b84a92a3aa29adab1514f1, State: Initialized, Role: FOLLOWER
I20260926 04:06:57.205921 19904 consensus_queue.cc:260] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.206203 19904 ts_tablet_manager.cc:1428] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260926 04:06:57.206270 19890 heartbeater.cc:503] Master 127.0.200.190:36527 was elected leader, sending a full tablet report...
W20260926 04:06:57.277163 19760 tablet.cc:2369] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:06:57.400204 19629 tablet.cc:2369] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 04:06:57.407506 19891 tablet.cc:2369] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:57.452313 19910 raft_consensus.cc:482] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:57.452447 19910 raft_consensus.cc:504] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.452874 19910 leader_election.cc:288] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a26fd9b6b84a92a3aa29adab1514f1 (127.0.200.131:39831), 69a78499161242fe80effcb646b98e97 (127.0.200.129:41233)
I20260926 04:06:57.456250 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.453171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41233 (local address 127.0.200.130:55103)
0926 04:06:57.453398 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.453407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.453458 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.453675 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.453676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.453679 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:57.453747 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.453748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.454463 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.454464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.455360 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.455362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.455953 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.455954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.456043 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.456047 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.456100 (+ 53us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.456136 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260926 04:06:57.456254 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.453339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:55103 (local address 127.0.200.129:41233)
0926 04:06:57.453389 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.453390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.453474 (+ 84us) server_negotiation.cc:407] Connection header received
0926 04:06:57.453505 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.453506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.453519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.453592 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:57.453838 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.454377 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.455459 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.455860 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.455917 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.455925 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.456132 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.456184 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:06:57.456319 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.453188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:39403 (local address 127.0.200.131:39831)
0926 04:06:57.453264 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.453266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.453280 (+ 14us) server_negotiation.cc:407] Connection header received
0926 04:06:57.453364 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.453365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.453377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.453442 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:57.453675 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.454223 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.455543 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.455949 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.456024 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.456033 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.456200 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.456239 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:06:57.456583 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.453104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:39831 (local address 127.0.200.130:39403)
0926 04:06:57.453214 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.453226 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:57.453278 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:57.453471 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:57.453472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:57.453474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:57.453555 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.453558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.454311 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.454312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.455415 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:57.455419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.456042 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.456043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:57.456121 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.456125 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:57.456180 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:06:57.456220 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 04:06:57.456727 19582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4309a0ccad874590b53b8d8a058ba916" candidate_uuid: "52a8440f88b34155995edea719cf910d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69a78499161242fe80effcb646b98e97" is_pre_election: true
I20260926 04:06:57.456904 19582 raft_consensus.cc:2454] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52a8440f88b34155995edea719cf910d in term 0.
I20260926 04:06:57.456924 19844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4309a0ccad874590b53b8d8a058ba916" candidate_uuid: "52a8440f88b34155995edea719cf910d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" is_pre_election: true
I20260926 04:06:57.457099 19844 raft_consensus.cc:2454] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52a8440f88b34155995edea719cf910d in term 0.
I20260926 04:06:57.457144 19647 leader_election.cc:302] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52a8440f88b34155995edea719cf910d, 69a78499161242fe80effcb646b98e97; no voters:
I20260926 04:06:57.457270 19910 raft_consensus.cc:2781] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:57.457346 19910 raft_consensus.cc:482] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:57.457391 19910 raft_consensus.cc:3037] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:57.458115 19910 raft_consensus.cc:504] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.458272 19910 leader_election.cc:288] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [CANDIDATE]: Term 1 election: Requested vote from peers b3a26fd9b6b84a92a3aa29adab1514f1 (127.0.200.131:39831), 69a78499161242fe80effcb646b98e97 (127.0.200.129:41233)
I20260926 04:06:57.458419 19582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4309a0ccad874590b53b8d8a058ba916" candidate_uuid: "52a8440f88b34155995edea719cf910d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69a78499161242fe80effcb646b98e97"
I20260926 04:06:57.458515 19582 raft_consensus.cc:3037] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:57.458637 19844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4309a0ccad874590b53b8d8a058ba916" candidate_uuid: "52a8440f88b34155995edea719cf910d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1"
I20260926 04:06:57.458736 19844 raft_consensus.cc:3037] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:57.459365 19582 raft_consensus.cc:2454] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52a8440f88b34155995edea719cf910d in term 1.
I20260926 04:06:57.459581 19844 raft_consensus.cc:2454] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52a8440f88b34155995edea719cf910d in term 1.
I20260926 04:06:57.459581 19647 leader_election.cc:302] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52a8440f88b34155995edea719cf910d, 69a78499161242fe80effcb646b98e97; no voters:
I20260926 04:06:57.459692 19910 raft_consensus.cc:2781] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:57.459857 19910 raft_consensus.cc:686] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 1 LEADER]: Becoming Leader. State: Replica: 52a8440f88b34155995edea719cf910d, State: Running, Role: LEADER
I20260926 04:06:57.459961 19910 consensus_queue.cc:237] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } }
I20260926 04:06:57.460747 19440 catalog_manager.cc:5495] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d reported cstate change: term changed from 0 to 1, leader changed from <none> to 52a8440f88b34155995edea719cf910d (127.0.200.130). New cstate: current_term: 1 leader_uuid: "52a8440f88b34155995edea719cf910d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52a8440f88b34155995edea719cf910d" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 34329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 } health_report { overall_health: UNKNOWN } } }
I20260926 04:06:57.477385 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.474803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.0.200.130:34329)
0926 04:06:57.474860 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:57.474862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:57.474872 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:57.474915 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:57.474916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:57.474946 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:57.475017 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:57.475234 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.475729 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.476448 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:57.476642 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:57.476697 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:57.476897 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:57.476988 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:57.477065 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:57.477282 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 04:06:57.477292 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 04:06:57.479568 19582 raft_consensus.cc:1261] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 1 FOLLOWER]: Refusing update from remote peer 52a8440f88b34155995edea719cf910d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:57.479560 19844 raft_consensus.cc:1261] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 1 FOLLOWER]: Refusing update from remote peer 52a8440f88b34155995edea719cf910d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:57.480000 19910 consensus_queue.cc:1035] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a26fd9b6b84a92a3aa29adab1514f1" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 39831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:57.480139 19914 consensus_queue.cc:1035] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [LEADER]: Connected to new peer: Peer: permanent_uuid: "69a78499161242fe80effcb646b98e97" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:57.483335 19918 mvcc.cc:205] Tried to move back new op lower bound from 7333460449193590784 to 7333460449117175808. Current Snapshot: MvccSnapshot[applied={T|T < 7333460449193590784}]
I20260926 04:06:57.485846 19439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:57.485975 19439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:57.486825 19439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bebdf290399247ddb88c59ed2845f064: Sending DeleteTablet for 3 replicas of tablet 4309a0ccad874590b53b8d8a058ba916
I20260926 04:06:57.487149 19693 tablet_service.cc:1514] Processing DeleteTablet for tablet 4309a0ccad874590b53b8d8a058ba916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:57 UTC) from {username='slave'} at 127.0.0.1:58872
I20260926 04:06:57.487219 19562 tablet_service.cc:1514] Processing DeleteTablet for tablet 4309a0ccad874590b53b8d8a058ba916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:57 UTC) from {username='slave'} at 127.0.0.1:42826
I20260926 04:06:57.487381 19824 tablet_service.cc:1514] Processing DeleteTablet for tablet 4309a0ccad874590b53b8d8a058ba916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:57 UTC) from {username='slave'} at 127.0.0.1:41518
I20260926 04:06:57.487493 19928 tablet_replica.cc:330] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97: stopping tablet replica
I20260926 04:06:57.487601 19928 raft_consensus.cc:2229] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:57.487732 19928 raft_consensus.cc:2258] T 4309a0ccad874590b53b8d8a058ba916 P 69a78499161242fe80effcb646b98e97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:57.487785 19927 tablet_replica.cc:330] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: stopping tablet replica
I20260926 04:06:57.487883 19927 raft_consensus.cc:2229] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:57.488019 19927 raft_consensus.cc:2258] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:57.488097 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19500
I20260926 04:06:57.489275 19927 ts_tablet_manager.cc:1905] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:57.487701 19929 tablet_replica.cc:330] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: stopping tablet replica
I20260926 04:06:57.491493 19927 ts_tablet_manager.cc:1918] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:57.491590 19927 log.cc:1196] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/4309a0ccad874590b53b8d8a058ba916
I20260926 04:06:57.491591 19929 raft_consensus.cc:2229] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:57.491780 19929 raft_consensus.cc:2258] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:57.491933 19927 ts_tablet_manager.cc:1939] T 4309a0ccad874590b53b8d8a058ba916 P 52a8440f88b34155995edea719cf910d: Deleting consensus metadata
I20260926 04:06:57.492324 19929 ts_tablet_manager.cc:1905] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:57.493283 19424 catalog_manager.cc:4841] TS 52a8440f88b34155995edea719cf910d (127.0.200.130:34329): tablet 4309a0ccad874590b53b8d8a058ba916 (table test-table [id=f36f94e0bbb248b6ba9b74a3da44b83b]) successfully deleted
I20260926 04:06:57.493784 19929 ts_tablet_manager.cc:1918] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:57.493870 19929 log.cc:1196] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/4309a0ccad874590b53b8d8a058ba916
I20260926 04:06:57.494096 19929 ts_tablet_manager.cc:1939] T 4309a0ccad874590b53b8d8a058ba916 P b3a26fd9b6b84a92a3aa29adab1514f1: Deleting consensus metadata
I20260926 04:06:57.494501 19426 catalog_manager.cc:4841] TS b3a26fd9b6b84a92a3aa29adab1514f1 (127.0.200.131:39831): tablet 4309a0ccad874590b53b8d8a058ba916 (table test-table [id=f36f94e0bbb248b6ba9b74a3da44b83b]) successfully deleted
W20260926 04:06:57.495157 19425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:41233 (error 108)
I20260926 04:06:57.495443 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19631
I20260926 04:06:57.495638 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:57.495493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:41233 (local address 127.0.0.1:42830)
0926 04:06:57.495564 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:57.495600 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 04:06:57.497112 19425 catalog_manager.cc:4584] TS 69a78499161242fe80effcb646b98e97 (127.0.200.129:41233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4309a0ccad874590b53b8d8a058ba916: Network error: Client connection negotiation failed: client connection to 127.0.200.129:41233: connect: Connection refused (error 111)
I20260926 04:06:57.500576 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19762
I20260926 04:06:57.505072 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19409
2026-09-26T04:06:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (915 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:06:57 dist-test-slave-j8c1 krb5kdc[19933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:06:57 dist-test-slave-j8c1 krb5kdc[19933](info): set up 2 sockets
Sep 26 04:06:57 dist-test-slave-j8c1 krb5kdc[19933](info): commencing operation
krb5kdc: starting...
W20260926 04:06:58.542841 802 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:06:58 dist-test-slave-j8c1 krb5kdc[19933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:06:58.666623 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43643
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:58.747352 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:58.747581 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:58.747604 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:58.748795 19946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:58.748842 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:58.748855 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:58.748867 19946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:58.748879 19946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:58.750423 19946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:58.750912 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:58.751129 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:58.756189 19951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:58.756193 19952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:58.756193 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:58.756283 19946 server_base.cc:1034] running on GCE node
I20260926 04:06:58.756631 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:58.756942 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:58.758114 19946 hybrid_clock.cc:648] HybridClock initialized: now 1790395618758093 us; error 40 us; skew 500 ppm
Sep 26 04:06:58 dist-test-slave-j8c1 krb5kdc[19933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:58.760717 19946 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:06:58.761278 19946 webserver.cc:460] Webserver started at http://127.0.200.190:41559/ using document root <none> and password file <none>
I20260926 04:06:58.761523 19946 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:58.761595 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:58.761746 19946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:58.762928 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "dd13b2885e354cf0970d4e9b55900289"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "48f9d0d10ce41875dc800d131c5823f7"
server_key_iv: "6c4ea09911872853bf688f04daa3ea41"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.763295 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "dd13b2885e354cf0970d4e9b55900289"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "48f9d0d10ce41875dc800d131c5823f7"
server_key_iv: "6c4ea09911872853bf688f04daa3ea41"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.764669 19946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:58.765420 19961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:58.765677 19946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:58.765789 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "dd13b2885e354cf0970d4e9b55900289"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "48f9d0d10ce41875dc800d131c5823f7"
server_key_iv: "6c4ea09911872853bf688f04daa3ea41"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.765884 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:58.774442 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:58.776449 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:58.776561 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:58.780859 19946 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43643
I20260926 04:06:58.780939 20013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43643 every 8 connection(s)
I20260926 04:06:58.781322 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:58.782255 20014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:58.784684 20014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: Bootstrap starting.
I20260926 04:06:58.785248 20014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:58.785486 20014 log.cc:824] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:58.786197 20014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: No bootstrap required, opened a new log
I20260926 04:06:58.787856 20014 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } }
I20260926 04:06:58.787998 20014 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:58.788019 20014 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13b2885e354cf0970d4e9b55900289, State: Initialized, Role: FOLLOWER
I20260926 04:06:58.788169 20014 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } }
I20260926 04:06:58.788264 20014 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:58.788288 20014 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:58.788323 20014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:58.788821 20014 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } }
I20260926 04:06:58.788908 20014 leader_election.cc:302] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd13b2885e354cf0970d4e9b55900289; no voters:
I20260926 04:06:58.789060 20014 leader_election.cc:288] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:58.789099 20017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:58.789218 20017 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [term 1 LEADER]: Becoming Leader. State: Replica: dd13b2885e354cf0970d4e9b55900289, State: Running, Role: LEADER
I20260926 04:06:58.789297 20017 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } }
I20260926 04:06:58.789341 20014 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:58.789728 20018 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd13b2885e354cf0970d4e9b55900289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } } }
I20260926 04:06:58.789836 20018 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:58.789999 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd13b2885e354cf0970d4e9b55900289. Latest consensus state: current_term: 1 leader_uuid: "dd13b2885e354cf0970d4e9b55900289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd13b2885e354cf0970d4e9b55900289" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43643 } } }
I20260926 04:06:58.790143 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:58.790382 20025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:58.790999 20025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:58.791436 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 19946
I20260926 04:06:58.791539 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:58.791769 802 external_mini_cluster.cc:1428] Setting key 62d3fafb26ce325ff6aa2739367209dd
I20260926 04:06:58.792677 20025 catalog_manager.cc:1347] Generated new cluster ID: 005cf917e36a4a998f62b9d2b59ba422
I20260926 04:06:58.792742 20025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:06:58 dist-test-slave-j8c1 krb5kdc[19933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:58.801754 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:58.792218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.0.200.190:43643)
0926 04:06:58.792398 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:58.792408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:58.792849 (+ 441us) server_negotiation.cc:407] Connection header received
0926 04:06:58.793011 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:58.793017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:58.793061 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:58.793198 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:58.793834 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.794832 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.795581 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.795998 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.796116 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:58.798447 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:58.798451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:58.798451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:58.798471 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:58.800217 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:58.800588 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:58.800591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:58.800591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:58.800695 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:58.800904 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:58.800905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:58.800905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:58.801153 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:58.801325 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:58.801422 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 04:06:58.801570 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260926 04:06:58.806438 20025 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:58.807008 20025 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:58.811421 20025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: Generated new TSK 0
I20260926 04:06:58.811605 20025 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.200.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
Entry for principal kudu/127.0.200.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab.
I20260926 04:06:58.846813 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:58.928905 20040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:58.929167 20040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:58.929221 20040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:58.930574 20040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:58.930652 20040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:58.930701 20040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:58.932307 20040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.129.keytab
--principal=kudu/127.0.200.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:58.932601 20040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:58.932852 20040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:58.937943 20045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:58.937959 20048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:58.937963 20046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:58.938325 20040 server_base.cc:1034] running on GCE node
I20260926 04:06:58.938588 20040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:58.938865 20040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:58.940008 20040 hybrid_clock.cc:648] HybridClock initialized: now 1790395618939997 us; error 28 us; skew 500 ppm
Sep 26 04:06:58 dist-test-slave-j8c1 krb5kdc[19933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:58.942803 20040 init.cc:377] Logged in from keytab as kudu/127.0.200.129@KRBTEST.COM (short username kudu)
I20260926 04:06:58.943290 20040 webserver.cc:460] Webserver started at http://127.0.200.129:44485/ using document root <none> and password file <none>
I20260926 04:06:58.943521 20040 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:58.943565 20040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:58.943694 20040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:58.944756 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "3010e4773b864e35a5d0b59408209fcb"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "056aec9023792eb2dc3842b9116461de"
server_key_iv: "009e23d2619c226d4d31493eefaaef92"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.945127 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "3010e4773b864e35a5d0b59408209fcb"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "056aec9023792eb2dc3842b9116461de"
server_key_iv: "009e23d2619c226d4d31493eefaaef92"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.946466 20040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260926 04:06:58.947244 20055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:58.947467 20040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:58.947582 20040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "3010e4773b864e35a5d0b59408209fcb"
format_stamp: "Formatted at 2026-09-26 04:06:58 on dist-test-slave-j8c1"
server_key: "056aec9023792eb2dc3842b9116461de"
server_key_iv: "009e23d2619c226d4d31493eefaaef92"
server_key_version: "encryptionkey@0"
I20260926 04:06:58.947662 20040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:58.966660 20040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:58.969103 20040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:58.969261 20040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:58.969507 20040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:58.969815 20040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:58.969869 20040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:58.969913 20040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:58.969944 20040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:58.976020 20040 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:39015
I20260926 04:06:58.976070 20168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:39015 every 8 connection(s)
I20260926 04:06:58.976393 20040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:58.982127 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20040
I20260926 04:06:58.982223 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:58.982475 802 external_mini_cluster.cc:1428] Setting key 2f40c6ba09530498f61268933b4e4bf4
Sep 26 04:06:58 dist-test-slave-j8c1 krb5kdc[19933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.129@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:58.986317 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:58.977155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:56309 (local address 127.0.200.190:43643)
0926 04:06:58.977210 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:58.977213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:58.977423 (+ 210us) server_negotiation.cc:407] Connection header received
0926 04:06:58.978420 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:58.978422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:58.978441 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:58.978499 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:58.979191 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.979740 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.980751 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.980965 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.981053 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:58.983397 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:58.983399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:58.983399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:58.983415 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:58.984815 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:58.985243 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:58.985245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:58.985245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:58.985310 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:58.985614 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:58.985615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:58.985615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:58.985750 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:58.985825 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:58.986125 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 04:06:58.986223 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:06:58.986599 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:58.977107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43643 (local address 127.0.200.129:56309)
0926 04:06:58.977260 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:58.977313 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:58.978269 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:58.978589 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:58.978593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:58.978891 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:58.979084 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:58.979088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.979839 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.979849 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:58.980657 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:58.980659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:58.981065 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:58.981066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:58.981201 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:58.981703 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:58.981711 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:58.983286 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:58.984914 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:58.984922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:58.984924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:58.985164 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:58.985417 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:58.985418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:58.985418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:58.985538 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:58.985827 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:58.985835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:58.985985 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 04:06:58.986091 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 04:06:58.986261 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260926 04:06:58.987126 20169 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43643
I20260926 04:06:58.987226 20169 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:58.987429 20169 heartbeater.cc:511] Master 127.0.200.190:43643 requested a full tablet report, sending...
I20260926 04:06:58.988036 19978 ts_manager.cc:194] Registered new tserver with Master: 3010e4773b864e35a5d0b59408209fcb (127.0.200.129:39015)
I20260926 04:06:58.989053 19978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.129@KRBTEST.COM'} at 127.0.200.129:56309
WARNING: no policy specified for kudu/127.0.200.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
Entry for principal kudu/127.0.200.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab.
I20260926 04:06:59.009883 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:59.091840 20174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:59.092048 20174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:59.092072 20174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:59.093498 20174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:59.093547 20174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:59.093577 20174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:59.095177 20174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.130.keytab
--principal=kudu/127.0.200.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:59.095435 20174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:59.095674 20174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:59.100801 20179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.100854 20180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.100801 20182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:59.101073 20174 server_base.cc:1034] running on GCE node
I20260926 04:06:59.101337 20174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:59.101626 20174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:59.102823 20174 hybrid_clock.cc:648] HybridClock initialized: now 1790395619102808 us; error 26 us; skew 500 ppm
Sep 26 04:06:59 dist-test-slave-j8c1 krb5kdc[19933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:59.105408 20174 init.cc:377] Logged in from keytab as kudu/127.0.200.130@KRBTEST.COM (short username kudu)
I20260926 04:06:59.105845 20174 webserver.cc:460] Webserver started at http://127.0.200.130:38037/ using document root <none> and password file <none>
I20260926 04:06:59.106086 20174 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:59.106151 20174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:59.106300 20174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:59.107396 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "7d2fefd36ffd491e9b03ac2799d8088c"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "25a6433a2ffca7fa5d599af4416eb980"
server_key_iv: "059440ca365731f4a8f52e2babd6367e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.107762 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "7d2fefd36ffd491e9b03ac2799d8088c"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "25a6433a2ffca7fa5d599af4416eb980"
server_key_iv: "059440ca365731f4a8f52e2babd6367e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.109148 20174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:06:59.109936 20189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.110128 20174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:59.110241 20174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "7d2fefd36ffd491e9b03ac2799d8088c"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "25a6433a2ffca7fa5d599af4416eb980"
server_key_iv: "059440ca365731f4a8f52e2babd6367e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.110332 20174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:59.129645 20174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:59.131747 20174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:59.131922 20174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:59.132236 20174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:59.132571 20174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:59.132627 20174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.132668 20174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:59.132710 20174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.138800 20174 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:35479
I20260926 04:06:59.138856 20302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:35479 every 8 connection(s)
I20260926 04:06:59.139228 20174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:59.145208 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20174
I20260926 04:06:59.145359 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:59.145569 802 external_mini_cluster.cc:1428] Setting key 0f8c691005d68dd07773b0de6b4493aa
Sep 26 04:06:59 dist-test-slave-j8c1 krb5kdc[19933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.130@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:59.150058 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.140078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:37267 (local address 127.0.200.190:43643)
0926 04:06:59.140122 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.140125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.140359 (+ 234us) server_negotiation.cc:407] Connection header received
0926 04:06:59.141326 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.141327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.141339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.141410 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.142154 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.142769 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.143899 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.144088 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.144142 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.146828 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.146830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.146831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:59.146839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.148513 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.148975 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.148976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.148977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.149040 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.149321 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.149322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.149322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.149447 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.149521 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.149853 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.149953 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:06:59.150288 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.140043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43643 (local address 127.0.200.130:37267)
0926 04:06:59.140226 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.140271 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.141182 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.141521 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.141526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.141831 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:59.142060 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.142064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.142871 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.142872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.143812 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.143814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.144249 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.144250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.144368 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.145082 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:59.145091 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:59.146704 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:59.148628 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:59.148637 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:59.148639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:59.148890 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:59.149111 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:59.149111 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:59.149112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:59.149242 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:59.149526 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:59.149532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:59.149714 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.149833 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.150004 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260926 04:06:59.150862 20303 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43643
I20260926 04:06:59.150986 20303 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:59.151192 20303 heartbeater.cc:511] Master 127.0.200.190:43643 requested a full tablet report, sending...
I20260926 04:06:59.151669 19978 ts_manager.cc:194] Registered new tserver with Master: 7d2fefd36ffd491e9b03ac2799d8088c (127.0.200.130:35479)
I20260926 04:06:59.152081 19978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.130@KRBTEST.COM'} at 127.0.200.130:37267
WARNING: no policy specified for kudu/127.0.200.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
Entry for principal kudu/127.0.200.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab.
I20260926 04:06:59.174772 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:06:59.263386 20308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:59.263600 20308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:59.263625 20308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:59.264796 20308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:59.264843 20308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:59.264869 20308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:06:59.266448 20308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/krb5kdc/kudu_127.0.200.131.keytab
--principal=kudu/127.0.200.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:59.266762 20308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:59.267005 20308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:59.272233 20316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.272226 20313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.272226 20314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:59.272393 20308 server_base.cc:1034] running on GCE node
I20260926 04:06:59.272607 20308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:59.272884 20308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:59.274035 20308 hybrid_clock.cc:648] HybridClock initialized: now 1790395619274001 us; error 56 us; skew 500 ppm
Sep 26 04:06:59 dist-test-slave-j8c1 krb5kdc[19933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:06:59.276851 20308 init.cc:377] Logged in from keytab as kudu/127.0.200.131@KRBTEST.COM (short username kudu)
I20260926 04:06:59.277386 20308 webserver.cc:460] Webserver started at http://127.0.200.131:38143/ using document root <none> and password file <none>
I20260926 04:06:59.277633 20308 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:59.277679 20308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:59.277844 20308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:59.279114 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "61a7be4ddcc649e4bf82a09a5b8312f2"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "28f3d0f4f36152f5346d82445819242a"
server_key_iv: "121989abe653b159a977e047ec0516ad"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.279531 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "61a7be4ddcc649e4bf82a09a5b8312f2"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "28f3d0f4f36152f5346d82445819242a"
server_key_iv: "121989abe653b159a977e047ec0516ad"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.280869 20308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:59.281699 20323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.281915 20308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:59.281987 20308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "61a7be4ddcc649e4bf82a09a5b8312f2"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "28f3d0f4f36152f5346d82445819242a"
server_key_iv: "121989abe653b159a977e047ec0516ad"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.282073 20308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:59.294816 20308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:59.296983 20308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:59.297102 20308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:59.297412 20308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:59.297753 20308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:59.297811 20308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.297854 20308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:59.297883 20308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.304073 20308 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:44205
I20260926 04:06:59.304118 20436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:44205 every 8 connection(s)
I20260926 04:06:59.304447 20308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:06:59.309834 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20308
I20260926 04:06:59.309942 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:06:59.310192 802 external_mini_cluster.cc:1428] Setting key 02d9faded94b78df1e47a86e72330e00
Sep 26 04:06:59 dist-test-slave-j8c1 krb5kdc[19933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.131@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:06:59.314812 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.305199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:58949 (local address 127.0.200.190:43643)
0926 04:06:59.305244 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.305246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.305483 (+ 237us) server_negotiation.cc:407] Connection header received
0926 04:06:59.306387 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.306388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.306399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.306469 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.307138 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.307652 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.308772 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.308964 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.309030 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.311759 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.311761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.311761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:59.311770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.313352 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.313777 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.313779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.313779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.313841 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.314120 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.314121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.314121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.314251 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.314326 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.314629 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.314720 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:06:59.315013 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.305161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43643 (local address 127.0.200.131:58949)
0926 04:06:59.305319 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.305362 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.306293 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.306525 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.306528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.306852 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:59.307045 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.307050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.307756 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.307757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.308646 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.308649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.309050 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.309050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.309172 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.309731 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:06:59.309743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:59.311632 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:59.313437 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:59.313445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:59.313448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:59.313685 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:59.313926 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:06:59.313926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:06:59.313927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:06:59.314044 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:06:59.314317 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:59.314323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:59.314468 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.314607 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.314775 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260926 04:06:59.315541 20437 heartbeater.cc:347] Connected to a master server at 127.0.200.190:43643
I20260926 04:06:59.315639 20437 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:59.315891 20437 heartbeater.cc:511] Master 127.0.200.190:43643 requested a full tablet report, sending...
I20260926 04:06:59.316298 19978 ts_manager.cc:194] Registered new tserver with Master: 61a7be4ddcc649e4bf82a09a5b8312f2 (127.0.200.131:44205)
I20260926 04:06:59.316627 19978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.200.131@KRBTEST.COM'} at 127.0.200.131:58949
I20260926 04:06:59.322368 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:06:59.330072 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.323779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.0.200.190:43643)
0926 04:06:59.323828 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.323830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.323907 (+ 77us) server_negotiation.cc:407] Connection header received
0926 04:06:59.323963 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.323963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.323972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.324021 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.324629 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.325116 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.325870 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.326047 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.326100 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.327513 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.327515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.327515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:06:59.327521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.328916 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.329257 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.329257 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.329258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.329309 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:06:59.329545 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:06:59.329545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:06:59.329545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:06:59.329692 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.329769 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.329911 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.329969 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:06:59.331871 19978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:06:59.333448 19978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:06:59.340363 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.0.200.131:44205)
0926 04:06:59.336769 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.336773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.336783 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:59.336819 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.336821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.336835 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.336935 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:59.337053 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.337818 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.339237 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.339949 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340046 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.340118 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.340237 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.340276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":86,"threads_started":1}
I20260926 04:06:59.340355 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44205 (local address 127.0.0.1:54892)
0926 04:06:59.336497 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.336536 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.336602 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.336942 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.336943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.336946 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:59.336994 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.336996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.337928 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.337930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.339161 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.339164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.340049 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.340134 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.340148 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.340222 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.340277 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":55,"threads_started":1}
I20260926 04:06:59.340639 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35479 (local address 127.0.0.1:50206)
0926 04:06:59.336496 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.336537 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.336620 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.336849 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.336852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.336854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:59.336926 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.336929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.338436 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.338437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.339451 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.339453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.340394 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.340455 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.340459 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.340517 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.340567 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260926 04:06:59.340951 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.0.200.130:35479)
0926 04:06:59.336501 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.336506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.336597 (+ 91us) server_negotiation.cc:407] Connection header received
0926 04:06:59.336693 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.336695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.336712 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.336823 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:59.337184 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.338320 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.339548 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.340230 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340301 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.340350 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.340822 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.340867 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260926 04:06:59.341177 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:39015 (local address 127.0.0.1:40994)
0926 04:06:59.336681 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.336688 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.336748 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.337090 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.337091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.337093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:59.337156 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.337157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.338626 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.338627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.339695 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.339698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.340930 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.341011 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.341017 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.341072 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.341116 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":3,"thread_start_us":131,"threads_started":1}
I20260926 04:06:59.341109 20371 tablet_service.cc:1467] Processing CreateTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 (DEFAULT_TABLE table=test-table [id=152b75bf8e65453eac2cfe4361278586]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:59.341398 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.336350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.0.200.129:39015)
0926 04:06:59.336517 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.336521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.336752 (+ 231us) server_negotiation.cc:407] Connection header received
0926 04:06:59.336823 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.336825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.336841 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.337012 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:59.337233 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.338326 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.339781 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.340778 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.340840 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.340899 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.341280 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.341332 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260926 04:06:59.341521 20371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542d899c1a9f4e7bb83e2425bc932fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:59.341609 20235 tablet_service.cc:1467] Processing CreateTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 (DEFAULT_TABLE table=test-table [id=152b75bf8e65453eac2cfe4361278586]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:59.341885 20235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542d899c1a9f4e7bb83e2425bc932fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:59.341943 20103 tablet_service.cc:1467] Processing CreateTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 (DEFAULT_TABLE table=test-table [id=152b75bf8e65453eac2cfe4361278586]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:06:59.342170 20103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542d899c1a9f4e7bb83e2425bc932fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:59.344794 20454 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Bootstrap starting.
I20260926 04:06:59.345431 20455 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Bootstrap starting.
I20260926 04:06:59.345500 20454 tablet_bootstrap.cc:654] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:59.346078 20455 tablet_bootstrap.cc:654] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:59.346098 20454 log.cc:824] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:59.346662 20455 log.cc:824] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:59.346750 20456 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Bootstrap starting.
I20260926 04:06:59.347012 20454 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: No bootstrap required, opened a new log
I20260926 04:06:59.347086 20454 ts_tablet_manager.cc:1397] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:59.347298 20455 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: No bootstrap required, opened a new log
I20260926 04:06:59.347342 20455 ts_tablet_manager.cc:1397] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:06:59.347411 20456 tablet_bootstrap.cc:654] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:59.347957 20456 log.cc:824] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:59.348752 20456 tablet_bootstrap.cc:492] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: No bootstrap required, opened a new log
I20260926 04:06:59.348809 20456 ts_tablet_manager.cc:1397] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:06:59.348845 20455 raft_consensus.cc:348] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.348974 20455 raft_consensus.cc:374] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:59.348994 20455 raft_consensus.cc:729] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2fefd36ffd491e9b03ac2799d8088c, State: Initialized, Role: FOLLOWER
I20260926 04:06:59.349124 20455 consensus_queue.cc:260] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.349283 20454 raft_consensus.cc:348] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.349416 20455 ts_tablet_manager.cc:1428] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 04:06:59.349445 20454 raft_consensus.cc:374] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:59.349478 20454 raft_consensus.cc:729] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3010e4773b864e35a5d0b59408209fcb, State: Initialized, Role: FOLLOWER
I20260926 04:06:59.349561 20303 heartbeater.cc:503] Master 127.0.200.190:43643 was elected leader, sending a full tablet report...
I20260926 04:06:59.349602 20454 consensus_queue.cc:260] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.349850 20454 ts_tablet_manager.cc:1428] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:06:59.349979 20169 heartbeater.cc:503] Master 127.0.200.190:43643 was elected leader, sending a full tablet report...
I20260926 04:06:59.351155 20456 raft_consensus.cc:348] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.351279 20456 raft_consensus.cc:374] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:59.351300 20456 raft_consensus.cc:729] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a7be4ddcc649e4bf82a09a5b8312f2, State: Initialized, Role: FOLLOWER
I20260926 04:06:59.351370 20456 consensus_queue.cc:260] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.351620 20437 heartbeater.cc:503] Master 127.0.200.190:43643 was elected leader, sending a full tablet report...
I20260926 04:06:59.351604 20456 ts_tablet_manager.cc:1428] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260926 04:06:59.389941 20304 tablet.cc:2369] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:59.473176 20462 raft_consensus.cc:482] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:06:59.473315 20462 raft_consensus.cc:504] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.473723 20462 leader_election.cc:288] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3010e4773b864e35a5d0b59408209fcb (127.0.200.129:39015), 7d2fefd36ffd491e9b03ac2799d8088c (127.0.200.130:35479)
W20260926 04:06:59.477087 20170 tablet.cc:2369] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 04:06:59.477913 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.474013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:59115 (local address 127.0.200.130:35479)
0926 04:06:59.474054 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.474057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.474269 (+ 212us) server_negotiation.cc:407] Connection header received
0926 04:06:59.474299 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.474300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.474311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.474361 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:59.474518 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.475246 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.476869 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.477492 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.477586 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.477596 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.477798 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.477839 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:06:59.477913 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.473989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:39967 (local address 127.0.200.129:39015)
0926 04:06:59.474040 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.474042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.474053 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:06:59.474131 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.474132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.474142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.474205 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:06:59.474409 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.475214 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.476868 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.477531 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.477597 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.477604 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.477796 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.477840 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 04:06:59.477947 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.473933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:39015 (local address 127.0.200.131:39967)
0926 04:06:59.473995 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.474009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.474057 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.474237 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.474239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.474241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:59.474313 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.474315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.475331 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.475332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.476728 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.476731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.477613 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.477614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.477715 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.477718 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.477773 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.477831 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":103}
I20260926 04:06:59.477953 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.473950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:35479 (local address 127.0.200.131:59115)
0926 04:06:59.474153 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.474164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.474244 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.474385 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.474386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.474388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:06:59.474449 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.474450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.475331 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.475332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.476728 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.476731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.477613 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.477614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.477704 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.477709 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.477773 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.477831 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":78,"thread_start_us":152,"threads_started":1}
I20260926 04:06:59.478359 20123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542d899c1a9f4e7bb83e2425bc932fe6" candidate_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3010e4773b864e35a5d0b59408209fcb" is_pre_election: true
I20260926 04:06:59.478448 20257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542d899c1a9f4e7bb83e2425bc932fe6" candidate_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" is_pre_election: true
I20260926 04:06:59.478531 20123 raft_consensus.cc:2454] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a7be4ddcc649e4bf82a09a5b8312f2 in term 0.
I20260926 04:06:59.478652 20257 raft_consensus.cc:2454] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a7be4ddcc649e4bf82a09a5b8312f2 in term 0.
I20260926 04:06:59.478796 20324 leader_election.cc:302] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3010e4773b864e35a5d0b59408209fcb, 61a7be4ddcc649e4bf82a09a5b8312f2; no voters:
I20260926 04:06:59.478978 20462 raft_consensus.cc:2781] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:06:59.479029 20462 raft_consensus.cc:482] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:06:59.479045 20462 raft_consensus.cc:3037] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:59.479709 20462 raft_consensus.cc:504] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.479867 20462 leader_election.cc:288] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [CANDIDATE]: Term 1 election: Requested vote from peers 3010e4773b864e35a5d0b59408209fcb (127.0.200.129:39015), 7d2fefd36ffd491e9b03ac2799d8088c (127.0.200.130:35479)
I20260926 04:06:59.480028 20123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542d899c1a9f4e7bb83e2425bc932fe6" candidate_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3010e4773b864e35a5d0b59408209fcb"
I20260926 04:06:59.480048 20257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542d899c1a9f4e7bb83e2425bc932fe6" candidate_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2fefd36ffd491e9b03ac2799d8088c"
I20260926 04:06:59.480124 20257 raft_consensus.cc:3037] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:59.480127 20123 raft_consensus.cc:3037] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:59.480866 20123 raft_consensus.cc:2454] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a7be4ddcc649e4bf82a09a5b8312f2 in term 1.
I20260926 04:06:59.480875 20257 raft_consensus.cc:2454] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a7be4ddcc649e4bf82a09a5b8312f2 in term 1.
I20260926 04:06:59.481078 20325 leader_election.cc:302] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a7be4ddcc649e4bf82a09a5b8312f2, 7d2fefd36ffd491e9b03ac2799d8088c; no voters:
I20260926 04:06:59.481192 20462 raft_consensus.cc:2781] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:59.481379 20462 raft_consensus.cc:686] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 1 LEADER]: Becoming Leader. State: Replica: 61a7be4ddcc649e4bf82a09a5b8312f2, State: Running, Role: LEADER
I20260926 04:06:59.481496 20462 consensus_queue.cc:237] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } }
I20260926 04:06:59.482499 19977 catalog_manager.cc:5495] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61a7be4ddcc649e4bf82a09a5b8312f2 (127.0.200.131). New cstate: current_term: 1 leader_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61a7be4ddcc649e4bf82a09a5b8312f2" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 44205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 } health_report { overall_health: UNKNOWN } } }
I20260926 04:06:59.523777 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.521223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.0.200.131:44205)
0926 04:06:59.521306 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.521308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.521318 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:06:59.521363 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.521364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.521382 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.521450 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:06:59.521594 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.522060 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.522949 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.523151 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.523237 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.523425 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:06:59.523523 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:06:59.523618 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.523641 (+ 23us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.523663 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 04:06:59.525712 20123 raft_consensus.cc:1261] T 542d899c1a9f4e7bb83e2425bc932fe6 P 3010e4773b864e35a5d0b59408209fcb [term 1 FOLLOWER]: Refusing update from remote peer 61a7be4ddcc649e4bf82a09a5b8312f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:59.525748 20257 raft_consensus.cc:1261] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 1 FOLLOWER]: Refusing update from remote peer 61a7be4ddcc649e4bf82a09a5b8312f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:06:59.526155 20464 consensus_queue.cc:1035] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d2fefd36ffd491e9b03ac2799d8088c" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:59.526321 20462 consensus_queue.cc:1035] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3010e4773b864e35a5d0b59408209fcb" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:06:59.528754 20467 mvcc.cc:205] Tried to move back new op lower bound from 7333460457574547456 to 7333460457397280768. Current Snapshot: MvccSnapshot[applied={T|T < 7333460457574547456}]
I20260926 04:06:59.529024 20469 mvcc.cc:205] Tried to move back new op lower bound from 7333460457574547456 to 7333460457397280768. Current Snapshot: MvccSnapshot[applied={T|T < 7333460457574547456}]
I20260926 04:06:59.530077 20468 mvcc.cc:205] Tried to move back new op lower bound from 7333460457574547456 to 7333460457397280768. Current Snapshot: MvccSnapshot[applied={T|T < 7333460457574547456 or (T in {7333460457574547456})}]
I20260926 04:06:59.531144 19977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:59.531244 19977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:06:59.532054 19977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd13b2885e354cf0970d4e9b55900289: Sending DeleteTablet for 3 replicas of tablet 542d899c1a9f4e7bb83e2425bc932fe6
I20260926 04:06:59.532423 20371 tablet_service.cc:1514] Processing DeleteTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:59 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:54892
I20260926 04:06:59.532476 20235 tablet_service.cc:1514] Processing DeleteTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:59 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:50206
I20260926 04:06:59.532565 20103 tablet_service.cc:1514] Processing DeleteTablet for tablet 542d899c1a9f4e7bb83e2425bc932fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:06:59 UTC) from {username='kudu', principal='kudu/127.0.200.190@KRBTEST.COM'} at 127.0.0.1:40994
I20260926 04:06:59.532665 20478 tablet_replica.cc:330] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: stopping tablet replica
I20260926 04:06:59.532774 20478 raft_consensus.cc:2229] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:06:59.532892 20478 raft_consensus.cc:2258] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:59.533056 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20040
I20260926 04:06:59.534593 20478 ts_tablet_manager.cc:1905] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:06:59.536188 20478 ts_tablet_manager.cc:1918] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:59.536285 20478 log.cc:1196] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/542d899c1a9f4e7bb83e2425bc932fe6
I20260926 04:06:59.536544 20478 ts_tablet_manager.cc:1939] T 542d899c1a9f4e7bb83e2425bc932fe6 P 7d2fefd36ffd491e9b03ac2799d8088c: Deleting consensus metadata
I20260926 04:06:59.536991 19963 catalog_manager.cc:4841] TS 7d2fefd36ffd491e9b03ac2799d8088c (127.0.200.130:35479): tablet 542d899c1a9f4e7bb83e2425bc932fe6 (table test-table [id=152b75bf8e65453eac2cfe4361278586]) successfully deleted
I20260926 04:06:59.538652 20477 tablet_replica.cc:330] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: stopping tablet replica
I20260926 04:06:59.538806 20477 raft_consensus.cc:2229] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:06:59.538924 20477 raft_consensus.cc:2258] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:06:59.539348 20477 ts_tablet_manager.cc:1905] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 04:06:59.539978 19962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:39015 (error 108)
I20260926 04:06:59.540522 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.540380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:39015 (local address 127.0.0.1:41010)
0926 04:06:59.540448 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.540486 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:39015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 04:06:59.540655 19962 catalog_manager.cc:4584] TS 3010e4773b864e35a5d0b59408209fcb (127.0.200.129:39015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 542d899c1a9f4e7bb83e2425bc932fe6: Network error: Client connection negotiation failed: client connection to 127.0.200.129:39015: connect: Connection refused (error 111)
I20260926 04:06:59.541100 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20174
I20260926 04:06:59.546496 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20308
I20260926 04:06:59.546519 20477 ts_tablet_manager.cc:1918] T 542d899c1a9f4e7bb83e2425bc932fe6 P 61a7be4ddcc649e4bf82a09a5b8312f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:06:59.551796 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 19946
I20260926 04:06:59.551891 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.551753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:44205 (local address 127.0.0.1:54912)
0926 04:06:59.551819 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.551847 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.131:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
2026-09-26T04:06:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2047 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T04:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:06:59Z Disabled control of system clock
I20260926 04:06:59.579983 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33189
--webserver_interface=127.0.200.190
--webserver_port=0
--builtin_ntp_servers=127.0.200.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:59.662441 20484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:59.662715 20484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:59.662737 20484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:59.663856 20484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:06:59.663892 20484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:59.663904 20484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:59.663916 20484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:06:59.663929 20484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:06:59.665216 20484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:33189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:59.665406 20484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:59.665643 20484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:59.670447 20490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.670511 20489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.670475 20492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:59.670747 20484 server_base.cc:1034] running on GCE node
I20260926 04:06:59.670964 20484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:59.671219 20484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:59.672367 20484 hybrid_clock.cc:648] HybridClock initialized: now 1790395619672348 us; error 35 us; skew 500 ppm
I20260926 04:06:59.673640 20484 webserver.cc:460] Webserver started at http://127.0.200.190:41427/ using document root <none> and password file <none>
I20260926 04:06:59.673893 20484 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:59.673942 20484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:59.674114 20484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:59.675253 20484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "7367c8669f4e4b47b36e290d1b665ab9"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "8b57324a801456a59cdb6eaa706cfaee"
server_key_iv: "5976ccdb801d6a123bbd8601145afa49"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.675616 20484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "7367c8669f4e4b47b36e290d1b665ab9"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "8b57324a801456a59cdb6eaa706cfaee"
server_key_iv: "5976ccdb801d6a123bbd8601145afa49"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.676921 20484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:59.677664 20498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.677837 20484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:06:59.677942 20484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "7367c8669f4e4b47b36e290d1b665ab9"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "8b57324a801456a59cdb6eaa706cfaee"
server_key_iv: "5976ccdb801d6a123bbd8601145afa49"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.678056 20484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:59.690941 20484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:59.691284 20484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:59.691442 20484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:59.695529 20484 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:33189
I20260926 04:06:59.695580 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:33189 every 8 connection(s)
I20260926 04:06:59.695940 20484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:06:59.696900 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:06:59.699204 20551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: Bootstrap starting.
I20260926 04:06:59.699784 20551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: Neither blocks nor log segments found. Creating new log.
I20260926 04:06:59.700101 20551 log.cc:824] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: Log is configured to *not* fsync() on all Append() calls
I20260926 04:06:59.700786 20551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: No bootstrap required, opened a new log
I20260926 04:06:59.702375 20551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } }
I20260926 04:06:59.702513 20551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:06:59.702594 20551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7367c8669f4e4b47b36e290d1b665ab9, State: Initialized, Role: FOLLOWER
I20260926 04:06:59.702782 20551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } }
I20260926 04:06:59.702867 20551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:06:59.702909 20551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:06:59.702961 20551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:06:59.703488 20551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } }
I20260926 04:06:59.703606 20551 leader_election.cc:302] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7367c8669f4e4b47b36e290d1b665ab9; no voters:
I20260926 04:06:59.703797 20551 leader_election.cc:288] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:06:59.703876 20554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:06:59.704059 20554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 7367c8669f4e4b47b36e290d1b665ab9, State: Running, Role: LEADER
I20260926 04:06:59.704071 20551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:06:59.704190 20554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } }
I20260926 04:06:59.704344 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20484
I20260926 04:06:59.704444 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:06:59.704650 802 external_mini_cluster.cc:1428] Setting key a17d1860aa3e7c8fb6f144805a46d0c4
I20260926 04:06:59.705202 20556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7367c8669f4e4b47b36e290d1b665ab9. Latest consensus state: current_term: 1 leader_uuid: "7367c8669f4e4b47b36e290d1b665ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } } }
I20260926 04:06:59.705295 20556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [sys.catalog]: This master's current role is: LEADER
I20260926 04:06:59.705487 20555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7367c8669f4e4b47b36e290d1b665ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7367c8669f4e4b47b36e290d1b665ab9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 33189 } } }
I20260926 04:06:59.705555 20555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9 [sys.catalog]: This master's current role is: LEADER
W20260926 04:06:59.706132 20572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:06:59.706231 20572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:06:59.706308 20569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:06:59.706466 20569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:06:59.708101 20569 catalog_manager.cc:1347] Generated new cluster ID: d43485fcb23c42eaabb03b3ad529428b
I20260926 04:06:59.708154 20569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:06:59.711179 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.705296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.0.200.190:33189)
0926 04:06:59.705758 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.705768 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.705805 (+ 37us) server_negotiation.cc:407] Connection header received
0926 04:06:59.705927 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.705936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.706008 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.706159 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.706404 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.707710 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.708822 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.709465 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.709606 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.710419 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.710423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.710423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:59.710674 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.710715 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:59.710730 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.710802 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.710880 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.710959 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":90,"threads_started":1}
I20260926 04:06:59.718832 20569 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:06:59.719535 20569 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:06:59.740909 20569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: Generated new TSK 0
I20260926 04:06:59.741153 20569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:06:59.742525 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.129:0
--local_ip_for_outbound_sockets=127.0.200.129
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--builtin_ntp_servers=127.0.200.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:06:59.824494 20575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:59.824707 20575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:59.824728 20575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:59.825865 20575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:59.825905 20575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:59.825932 20575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.129
I20260926 04:06:59.827342 20575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.200.129
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.129
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:59.827592 20575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:59.827894 20575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:06:59.833964 20583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.833999 20581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.833969 20580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:59.834197 20575 server_base.cc:1034] running on GCE node
I20260926 04:06:59.834414 20575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:59.834721 20575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:59.835878 20575 hybrid_clock.cc:648] HybridClock initialized: now 1790395619835863 us; error 34 us; skew 500 ppm
I20260926 04:06:59.837139 20575 webserver.cc:460] Webserver started at http://127.0.200.129:44831/ using document root <none> and password file <none>
I20260926 04:06:59.837374 20575 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:59.837442 20575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:59.837606 20575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:59.838738 20575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data/instance:
uuid: "bcd770a3698b4d208ea4b5542919243f"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "aff7360e66dec1f3d2cfb22f7cfdd182"
server_key_iv: "d159ab89b8bc184cddd257abc452d79e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.839097 20575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal/instance:
uuid: "bcd770a3698b4d208ea4b5542919243f"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "aff7360e66dec1f3d2cfb22f7cfdd182"
server_key_iv: "d159ab89b8bc184cddd257abc452d79e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.840502 20575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:06:59.841252 20589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.841495 20575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:06:59.841591 20575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal
uuid: "bcd770a3698b4d208ea4b5542919243f"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "aff7360e66dec1f3d2cfb22f7cfdd182"
server_key_iv: "d159ab89b8bc184cddd257abc452d79e"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.841671 20575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:59.857679 20575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:59.857931 20575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:59.858042 20575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:59.858217 20575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:59.858563 20575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:59.858599 20575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.858623 20575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:59.858666 20575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.864224 20575 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.129:43545
I20260926 04:06:59.864296 20702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.129:43545 every 8 connection(s)
I20260926 04:06:59.864642 20575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/data/info.pb
I20260926 04:06:59.867781 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20575
I20260926 04:06:59.867871 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-0/wal/instance
I20260926 04:06:59.868096 802 external_mini_cluster.cc:1428] Setting key 85dd1c244cf4ebd9f8e5980556d7fba8
I20260926 04:06:59.869166 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.130:0
--local_ip_for_outbound_sockets=127.0.200.130
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--builtin_ntp_servers=127.0.200.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 04:06:59.870591 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.865425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.129:36763 (local address 127.0.200.190:33189)
0926 04:06:59.865469 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.865471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.865682 (+ 211us) server_negotiation.cc:407] Connection header received
0926 04:06:59.866206 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.866207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.866222 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.866267 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.867043 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.867552 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.868827 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.869190 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.869272 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.870255 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.870256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.870257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:59.870262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.870278 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:59.870282 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.870326 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.870434 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.870471 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:06:59.870792 20705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.865379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33189 (local address 127.0.200.129:36763)
0926 04:06:59.865561 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.865601 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.866103 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.866354 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.866357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.866365 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:59.866949 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.866954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.867662 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.867664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.868722 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.868729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.869298 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.869300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.869489 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.870111 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:59.870111 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:59.870139 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:59.870161 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:59.870337 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:59.870341 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:59.870345 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.870435 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.870535 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260926 04:06:59.871200 20703 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33189
I20260926 04:06:59.871289 20703 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:59.871428 20703 heartbeater.cc:511] Master 127.0.200.190:33189 requested a full tablet report, sending...
I20260926 04:06:59.871901 20515 ts_manager.cc:194] Registered new tserver with Master: bcd770a3698b4d208ea4b5542919243f (127.0.200.129:43545)
I20260926 04:06:59.872910 20515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.129:36763
W20260926 04:06:59.949781 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:06:59.950021 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:06:59.950043 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:06:59.951259 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:06:59.951300 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:06:59.951326 20706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.130
I20260926 04:06:59.952702 20706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.200.130
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.130
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:06:59.952957 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:06:59.953258 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 04:06:59.958189 20706 server_base.cc:1034] running on GCE node
W20260926 04:06:59.958159 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.958165 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:06:59.958150 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:06:59.958506 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:06:59.958752 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:06:59.959890 20706 hybrid_clock.cc:648] HybridClock initialized: now 1790395619959876 us; error 30 us; skew 500 ppm
I20260926 04:06:59.961073 20706 webserver.cc:460] Webserver started at http://127.0.200.130:37009/ using document root <none> and password file <none>
I20260926 04:06:59.961297 20706 fs_manager.cc:359] Metadata directory not provided
I20260926 04:06:59.961340 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:06:59.961490 20706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:06:59.962527 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data/instance:
uuid: "4246bb7c6bac4e6b81a82240ed487052"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "5b0a8af0c5b0ccfa2bab2c3c0b79cdf8"
server_key_iv: "bfc671dbb6fd0a948cfbf2f9a7e79777"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.962951 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal/instance:
uuid: "4246bb7c6bac4e6b81a82240ed487052"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "5b0a8af0c5b0ccfa2bab2c3c0b79cdf8"
server_key_iv: "bfc671dbb6fd0a948cfbf2f9a7e79777"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.964205 20706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:59.964998 20720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.965214 20706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:06:59.965283 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal
uuid: "4246bb7c6bac4e6b81a82240ed487052"
format_stamp: "Formatted at 2026-09-26 04:06:59 on dist-test-slave-j8c1"
server_key: "5b0a8af0c5b0ccfa2bab2c3c0b79cdf8"
server_key_iv: "bfc671dbb6fd0a948cfbf2f9a7e79777"
server_key_version: "encryptionkey@0"
I20260926 04:06:59.965374 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:06:59.980420 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:06:59.980716 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:06:59.980851 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:06:59.981113 20706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:06:59.981441 20706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:06:59.981472 20706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.981494 20706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:06:59.981540 20706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:06:59.987217 20706 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.130:40741
I20260926 04:06:59.987342 20833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.130:40741 every 8 connection(s)
I20260926 04:06:59.987632 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/data/info.pb
I20260926 04:06:59.993072 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.988464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:52175 (local address 127.0.200.190:33189)
0926 04:06:59.988544 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0926 04:06:59.988547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:06:59.988763 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:06:59.989327 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:06:59.989328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:06:59.989341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:06:59.989395 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:06:59.990105 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.990636 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.991615 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.991835 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.991926 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.992799 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:06:59.992800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:06:59.992800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:06:59.992805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:06:59.992821 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:06:59.992825 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:06:59.992880 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:06:59.992962 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 04:06:59.992992 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:06:59.993306 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:06:59.988383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33189 (local address 127.0.200.130:52175)
0926 04:06:59.988640 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 04:06:59.988674 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:06:59.989210 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:06:59.989466 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:06:59.989470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:06:59.989480 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:06:59.990010 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.990014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.990736 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.990737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.991523 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:06:59.991525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:06:59.991942 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:06:59.991943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:06:59.992076 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:06:59.992647 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:06:59.992647 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:06:59.992675 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:06:59.992697 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:06:59.992883 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:06:59.992887 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:06:59.992891 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:06:59.992961 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 04:06:59.993062 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260926 04:06:59.993882 20834 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33189
I20260926 04:06:59.994000 20834 heartbeater.cc:464] Registering TS with master...
I20260926 04:06:59.994144 20834 heartbeater.cc:511] Master 127.0.200.190:33189 requested a full tablet report, sending...
I20260926 04:06:59.994511 20515 ts_manager.cc:194] Registered new tserver with Master: 4246bb7c6bac4e6b81a82240ed487052 (127.0.200.130:40741)
I20260926 04:06:59.994750 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20706
I20260926 04:06:59.994874 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal/instance
I20260926 04:06:59.995019 20515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.130:52175
I20260926 04:06:59.995095 802 external_mini_cluster.cc:1428] Setting key 7120a0daef9ae6d0018106162153e7d2
I20260926 04:06:59.995963 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.200.131:0
--local_ip_for_outbound_sockets=127.0.200.131
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--builtin_ntp_servers=127.0.200.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 04:07:00.077994 20837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:00.078214 20837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:00.078238 20837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:00.079429 20837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:00.079475 20837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:00.079501 20837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.200.131
I20260926 04:07:00.080817 20837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.200.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.200.131
--webserver_port=0
--tserver_master_addrs=127.0.200.190:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.200.131
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:00.081022 20837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:00.081274 20837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:00.086362 20845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:00.086383 20843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:00.086374 20842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:00.086736 20837 server_base.cc:1034] running on GCE node
I20260926 04:07:00.086994 20837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:00.087271 20837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:00.088431 20837 hybrid_clock.cc:648] HybridClock initialized: now 1790395620088408 us; error 41 us; skew 500 ppm
I20260926 04:07:00.089813 20837 webserver.cc:460] Webserver started at http://127.0.200.131:42863/ using document root <none> and password file <none>
I20260926 04:07:00.090081 20837 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:00.090129 20837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:00.090261 20837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:00.091439 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data/instance:
uuid: "968d5e3eaad047ff98e3fe63b7327827"
format_stamp: "Formatted at 2026-09-26 04:07:00 on dist-test-slave-j8c1"
server_key: "6474283da355d72e0d10e25efa36f4d2"
server_key_iv: "09305e43527fea11899184e6249c5778"
server_key_version: "encryptionkey@0"
I20260926 04:07:00.091802 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal/instance:
uuid: "968d5e3eaad047ff98e3fe63b7327827"
format_stamp: "Formatted at 2026-09-26 04:07:00 on dist-test-slave-j8c1"
server_key: "6474283da355d72e0d10e25efa36f4d2"
server_key_iv: "09305e43527fea11899184e6249c5778"
server_key_version: "encryptionkey@0"
I20260926 04:07:00.093166 20837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 04:07:00.093988 20851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:00.094211 20837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:00.094290 20837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal
uuid: "968d5e3eaad047ff98e3fe63b7327827"
format_stamp: "Formatted at 2026-09-26 04:07:00 on dist-test-slave-j8c1"
server_key: "6474283da355d72e0d10e25efa36f4d2"
server_key_iv: "09305e43527fea11899184e6249c5778"
server_key_version: "encryptionkey@0"
I20260926 04:07:00.094383 20837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:00.109200 20837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:00.109526 20837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:00.109675 20837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:00.109922 20837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 04:07:00.110265 20837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 04:07:00.110299 20837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:00.110321 20837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 04:07:00.110368 20837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:00.116389 20837 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.131:36443
I20260926 04:07:00.116436 20964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.131:36443 every 8 connection(s)
I20260926 04:07:00.116784 20837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/data/info.pb
I20260926 04:07:00.121192 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 20837
I20260926 04:07:00.121321 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal/instance
I20260926 04:07:00.121556 802 external_mini_cluster.cc:1428] Setting key 4e5e0217897ffd04273ac874d01cdef8
I20260926 04:07:00.122392 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.117538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.131:43925 (local address 127.0.200.190:33189)
0926 04:07:00.117584 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.117586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.117849 (+ 263us) server_negotiation.cc:407] Connection header received
0926 04:07:00.118334 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.118335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.118346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.118418 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:00.119182 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.119700 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.120740 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.120984 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.121082 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.122093 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:00.122094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:00.122095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:00.122099 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:00.122118 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:00.122123 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:00.122178 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.122290 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.122323 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:00.122695 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.117548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:33189 (local address 127.0.200.131:43925)
0926 04:07:00.117724 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.117758 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.118226 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.118470 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.118473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.118481 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:00.119071 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.119076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.119819 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.119820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.120642 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.120645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.121107 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.121108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.121225 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.121934 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 04:07:00.121934 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:00.121975 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 04:07:00.122006 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:00.122193 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:00.122198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:00.122202 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.122270 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.122427 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260926 04:07:00.123373 20965 heartbeater.cc:347] Connected to a master server at 127.0.200.190:33189
I20260926 04:07:00.123481 20965 heartbeater.cc:464] Registering TS with master...
I20260926 04:07:00.123638 20965 heartbeater.cc:511] Master 127.0.200.190:33189 requested a full tablet report, sending...
I20260926 04:07:00.123981 20515 ts_manager.cc:194] Registered new tserver with Master: 968d5e3eaad047ff98e3fe63b7327827 (127.0.200.131:36443)
I20260926 04:07:00.124311 20515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.200.131:43925
I20260926 04:07:00.134100 802 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 04:07:00.138803 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.135604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.0.200.190:33189)
0926 04:07:00.135691 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.135693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.135755 (+ 62us) server_negotiation.cc:407] Connection header received
0926 04:07:00.135833 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.135834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.135849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.135951 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:00.136128 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.136649 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.137409 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.137612 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.137692 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.138434 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:00.138435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:00.138436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:00.138441 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:00.138475 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:00.138479 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:00.138548 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.138668 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.138699 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 04:07:00.140658 20515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 04:07:00.142295 20515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 04:07:00.149005 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.130:40741 (local address 127.0.0.1:41150)
0926 04:07:00.144961 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.145005 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.145083 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.145423 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.145423 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.145426 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:07:00.145474 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.145476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.146541 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.146567 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.147905 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.147907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148653 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.148738 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148751 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.148822 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.148896 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":100,"threads_started":1}
I20260926 04:07:00.149143 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:36443 (local address 127.0.0.1:56412)
0926 04:07:00.145056 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.145065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.145105 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.145759 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.145760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.145762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:07:00.145800 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.145802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.147023 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.147024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.147924 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.147927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148912 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.148974 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148978 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.149035 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.149074 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":144,"threads_started":1}
I20260926 04:07:00.149012 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43545 (local address 127.0.0.1:43644)
0926 04:07:00.145043 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.145055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.145112 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.145339 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.145342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.145344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:07:00.145414 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.145416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.146768 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.146769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.147907 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.147909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148811 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.148863 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148865 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.148920 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.148953 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":7,"thread_start_us":90,"threads_started":1}
I20260926 04:07:00.149076 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.0.200.129:43545)
0926 04:07:00.145161 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.145166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.145176 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:07:00.145214 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.145216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.145237 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.145328 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:07:00.145509 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.146679 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.147999 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148723 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148793 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148875 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.148932 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.148979 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":116,"threads_started":1}
I20260926 04:07:00.149421 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.0.200.130:40741)
0926 04:07:00.145248 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.145252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.145260 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:07:00.145292 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.145293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.145305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.145418 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:07:00.145564 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.146449 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148029 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148549 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148653 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148760 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.149062 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.149327 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":80,"threads_started":1}
I20260926 04:07:00.149497 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.144838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56412 (local address 127.0.200.131:36443)
0926 04:07:00.145551 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.145556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.145565 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:00.145603 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.145604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.145618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.145692 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:07:00.145900 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.146936 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148038 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.148723 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.148793 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.148875 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.149335 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.149384 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":80,"threads_started":1}
I20260926 04:07:00.150027 20637 tablet_service.cc:1467] Processing CreateTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f (DEFAULT_TABLE table=test-table [id=8aec9caef506464bb0e97aa435e095c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:07:00.150066 20899 tablet_service.cc:1467] Processing CreateTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f (DEFAULT_TABLE table=test-table [id=8aec9caef506464bb0e97aa435e095c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:07:00.150066 20768 tablet_service.cc:1467] Processing CreateTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f (DEFAULT_TABLE table=test-table [id=8aec9caef506464bb0e97aa435e095c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 04:07:00.150318 20637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f824fbfcd8b4f8bba9559ac2cace87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:00.150394 20768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f824fbfcd8b4f8bba9559ac2cace87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:00.150394 20899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f824fbfcd8b4f8bba9559ac2cace87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:00.152994 20979 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: Bootstrap starting.
I20260926 04:07:00.153720 20979 tablet_bootstrap.cc:654] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:00.153705 20980 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Bootstrap starting.
I20260926 04:07:00.154045 20979 log.cc:824] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:00.154356 20980 tablet_bootstrap.cc:654] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:00.154315 20981 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Bootstrap starting.
I20260926 04:07:00.154687 20980 log.cc:824] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:00.155007 20981 tablet_bootstrap.cc:654] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:00.155210 20979 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: No bootstrap required, opened a new log
I20260926 04:07:00.155288 20979 ts_tablet_manager.cc:1397] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 04:07:00.155325 20981 log.cc:824] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:00.155476 20980 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: No bootstrap required, opened a new log
I20260926 04:07:00.155540 20980 ts_tablet_manager.cc:1397] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 04:07:00.156010 20981 tablet_bootstrap.cc:492] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: No bootstrap required, opened a new log
I20260926 04:07:00.156100 20981 ts_tablet_manager.cc:1397] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260926 04:07:00.157985 20979 raft_consensus.cc:348] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158043 20980 raft_consensus.cc:348] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158172 20979 raft_consensus.cc:374] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:00.158171 20980 raft_consensus.cc:374] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:00.158075 20981 raft_consensus.cc:348] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158208 20979 raft_consensus.cc:729] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd770a3698b4d208ea4b5542919243f, State: Initialized, Role: FOLLOWER
I20260926 04:07:00.158208 20980 raft_consensus.cc:729] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 968d5e3eaad047ff98e3fe63b7327827, State: Initialized, Role: FOLLOWER
I20260926 04:07:00.158232 20981 raft_consensus.cc:374] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:00.158267 20981 raft_consensus.cc:729] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4246bb7c6bac4e6b81a82240ed487052, State: Initialized, Role: FOLLOWER
I20260926 04:07:00.158360 20979 consensus_queue.cc:260] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158360 20980 consensus_queue.cc:260] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158368 20981 consensus_queue.cc:260] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.158646 20981 ts_tablet_manager.cc:1428] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260926 04:07:00.158658 20979 ts_tablet_manager.cc:1428] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:07:00.158720 20834 heartbeater.cc:503] Master 127.0.200.190:33189 was elected leader, sending a full tablet report...
I20260926 04:07:00.158797 20703 heartbeater.cc:503] Master 127.0.200.190:33189 was elected leader, sending a full tablet report...
I20260926 04:07:00.158901 20965 heartbeater.cc:503] Master 127.0.200.190:33189 was elected leader, sending a full tablet report...
I20260926 04:07:00.159055 20980 ts_tablet_manager.cc:1428] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 04:07:00.193054 20985 raft_consensus.cc:482] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 04:07:00.193183 20985 raft_consensus.cc:504] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.193624 20985 leader_election.cc:288] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 968d5e3eaad047ff98e3fe63b7327827 (127.0.200.131:36443), bcd770a3698b4d208ea4b5542919243f (127.0.200.129:43545)
I20260926 04:07:00.196971 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.193943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:44129 (local address 127.0.200.131:36443)
0926 04:07:00.193987 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.193989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.194000 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:07:00.194060 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.194061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.194070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.194144 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:07:00.194320 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.195084 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196168 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.196557 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196649 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.196658 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.196839 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.196897 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:00.196990 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.193860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.131:36443 (local address 127.0.200.130:44129)
0926 04:07:00.193919 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.193932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.193987 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.194158 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.194159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.194161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:07:00.194237 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.194239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.195178 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.195180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.196051 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.196054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.196646 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.196725 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.196728 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.196829 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.196876 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:07:00.197011 20988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.193876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43545 (local address 127.0.200.130:58735)
0926 04:07:00.194044 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.194054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:00.194139 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:00.194292 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:00.194293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:00.194295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 04:07:00.194362 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.194363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.195255 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.195256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.196042 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:00.196045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.196684 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:00.196769 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.196779 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 04:07:00.196852 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 04:07:00.196895 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260926 04:07:00.196997 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.193970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.200.130:58735 (local address 127.0.200.129:43545)
0926 04:07:00.194016 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.194019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.194125 (+ 106us) server_negotiation.cc:407] Connection header received
0926 04:07:00.194216 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.194217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.194229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.194289 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 04:07:00.194467 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.195169 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196146 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.196589 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.196671 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.196681 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.196875 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.196928 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:00.197343 20657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f824fbfcd8b4f8bba9559ac2cace87f" candidate_uuid: "4246bb7c6bac4e6b81a82240ed487052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd770a3698b4d208ea4b5542919243f" is_pre_election: true
I20260926 04:07:00.197449 20919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f824fbfcd8b4f8bba9559ac2cace87f" candidate_uuid: "4246bb7c6bac4e6b81a82240ed487052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968d5e3eaad047ff98e3fe63b7327827" is_pre_election: true
I20260926 04:07:00.197554 20657 raft_consensus.cc:2454] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4246bb7c6bac4e6b81a82240ed487052 in term 0.
I20260926 04:07:00.197573 20919 raft_consensus.cc:2454] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4246bb7c6bac4e6b81a82240ed487052 in term 0.
I20260926 04:07:00.197788 20721 leader_election.cc:302] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4246bb7c6bac4e6b81a82240ed487052, bcd770a3698b4d208ea4b5542919243f; no voters:
I20260926 04:07:00.197964 20985 raft_consensus.cc:2781] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 04:07:00.198027 20985 raft_consensus.cc:482] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 04:07:00.198045 20985 raft_consensus.cc:3037] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:00.198894 20985 raft_consensus.cc:504] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.199056 20985 leader_election.cc:288] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [CANDIDATE]: Term 1 election: Requested vote from peers 968d5e3eaad047ff98e3fe63b7327827 (127.0.200.131:36443), bcd770a3698b4d208ea4b5542919243f (127.0.200.129:43545)
I20260926 04:07:00.199201 20657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f824fbfcd8b4f8bba9559ac2cace87f" candidate_uuid: "4246bb7c6bac4e6b81a82240ed487052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd770a3698b4d208ea4b5542919243f"
I20260926 04:07:00.199201 20919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f824fbfcd8b4f8bba9559ac2cace87f" candidate_uuid: "4246bb7c6bac4e6b81a82240ed487052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968d5e3eaad047ff98e3fe63b7327827"
I20260926 04:07:00.199282 20919 raft_consensus.cc:3037] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:00.199282 20657 raft_consensus.cc:3037] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:00.200208 20919 raft_consensus.cc:2454] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4246bb7c6bac4e6b81a82240ed487052 in term 1.
I20260926 04:07:00.200208 20657 raft_consensus.cc:2454] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4246bb7c6bac4e6b81a82240ed487052 in term 1.
I20260926 04:07:00.200402 20721 leader_election.cc:302] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4246bb7c6bac4e6b81a82240ed487052, bcd770a3698b4d208ea4b5542919243f; no voters:
I20260926 04:07:00.200572 20985 raft_consensus.cc:2781] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:00.200762 20985 raft_consensus.cc:686] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 1 LEADER]: Becoming Leader. State: Replica: 4246bb7c6bac4e6b81a82240ed487052, State: Running, Role: LEADER
I20260926 04:07:00.200870 20985 consensus_queue.cc:237] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } }
I20260926 04:07:00.201754 20515 catalog_manager.cc:5495] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4246bb7c6bac4e6b81a82240ed487052 (127.0.200.130). New cstate: current_term: 1 leader_uuid: "4246bb7c6bac4e6b81a82240ed487052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4246bb7c6bac4e6b81a82240ed487052" member_type: VOTER last_known_addr { host: "127.0.200.130" port: 40741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260926 04:07:00.210873 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.208446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.0.200.130:40741)
0926 04:07:00.208506 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:00.208508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:00.208517 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:00.208561 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:00.208562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:00.208580 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:00.208648 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:00.208802 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.209303 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.210097 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:00.210289 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:00.210356 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:00.210484 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:00.210643 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:00.210710 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:00.210743 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 04:07:00.210761 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 04:07:00.212958 20657 raft_consensus.cc:1261] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 1 FOLLOWER]: Refusing update from remote peer 4246bb7c6bac4e6b81a82240ed487052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:00.213032 20919 raft_consensus.cc:1261] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 1 FOLLOWER]: Refusing update from remote peer 4246bb7c6bac4e6b81a82240ed487052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:00.213337 20985 consensus_queue.cc:1035] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd770a3698b4d208ea4b5542919243f" member_type: VOTER last_known_addr { host: "127.0.200.129" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:07:00.213519 20989 consensus_queue.cc:1035] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "968d5e3eaad047ff98e3fe63b7327827" member_type: VOTER last_known_addr { host: "127.0.200.131" port: 36443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:07:00.216015 20992 mvcc.cc:205] Tried to move back new op lower bound from 7333460460389068800 to 7333460460344098816. Current Snapshot: MvccSnapshot[applied={T|T < 7333460460389068800}]
I20260926 04:07:00.216357 20993 mvcc.cc:205] Tried to move back new op lower bound from 7333460460389068800 to 7333460460344098816. Current Snapshot: MvccSnapshot[applied={T|T < 7333460460389068800}]
I20260926 04:07:00.218771 20515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:07:00.218866 20515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 04:07:00.219642 20515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7367c8669f4e4b47b36e290d1b665ab9: Sending DeleteTablet for 3 replicas of tablet 0f824fbfcd8b4f8bba9559ac2cace87f
I20260926 04:07:00.219971 20768 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:07:00 UTC) from {username='slave'} at 127.0.0.1:41150
I20260926 04:07:00.220114 20899 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:07:00 UTC) from {username='slave'} at 127.0.0.1:56412
I20260926 04:07:00.220343 20637 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f824fbfcd8b4f8bba9559ac2cace87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 04:07:00 UTC) from {username='slave'} at 127.0.0.1:43644
I20260926 04:07:00.220352 21003 tablet_replica.cc:330] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: stopping tablet replica
I20260926 04:07:00.220508 21003 raft_consensus.cc:2229] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:07:00.220566 21004 tablet_replica.cc:330] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f: stopping tablet replica
I20260926 04:07:00.220682 21004 raft_consensus.cc:2229] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 04:07:00.220774 21002 tablet_replica.cc:330] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: stopping tablet replica
I20260926 04:07:00.220798 21004 raft_consensus.cc:2258] T 0f824fbfcd8b4f8bba9559ac2cace87f P bcd770a3698b4d208ea4b5542919243f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:07:00.220860 21002 raft_consensus.cc:2229] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 1 LEADER]: Raft consensus shutting down.
I20260926 04:07:00.220990 21002 raft_consensus.cc:2258] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:07:00.221052 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20575
I20260926 04:07:00.222473 21003 raft_consensus.cc:2258] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 04:07:00.222939 21002 ts_tablet_manager.cc:1905] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:07:00.223125 21003 ts_tablet_manager.cc:1905] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 04:07:00.224968 21003 ts_tablet_manager.cc:1918] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:07:00.225132 21003 log.cc:1196] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-2/wal/wals/0f824fbfcd8b4f8bba9559ac2cace87f
I20260926 04:07:00.225121 21002 ts_tablet_manager.cc:1918] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 04:07:00.225189 21002 log.cc:1196] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Deleting WAL directory at /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790395371830964-802-0/minicluster-data/ts-1/wal/wals/0f824fbfcd8b4f8bba9559ac2cace87f
I20260926 04:07:00.225454 21002 ts_tablet_manager.cc:1939] T 0f824fbfcd8b4f8bba9559ac2cace87f P 4246bb7c6bac4e6b81a82240ed487052: Deleting consensus metadata
I20260926 04:07:00.225608 21003 ts_tablet_manager.cc:1939] T 0f824fbfcd8b4f8bba9559ac2cace87f P 968d5e3eaad047ff98e3fe63b7327827: Deleting consensus metadata
I20260926 04:07:00.225931 20500 catalog_manager.cc:4841] TS 4246bb7c6bac4e6b81a82240ed487052 (127.0.200.130:40741): tablet 0f824fbfcd8b4f8bba9559ac2cace87f (table test-table [id=8aec9caef506464bb0e97aa435e095c1]) successfully deleted
I20260926 04:07:00.226126 20501 catalog_manager.cc:4841] TS 968d5e3eaad047ff98e3fe63b7327827 (127.0.200.131:36443): tablet 0f824fbfcd8b4f8bba9559ac2cace87f (table test-table [id=8aec9caef506464bb0e97aa435e095c1]) successfully deleted
W20260926 04:07:00.227698 20499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.200.129:43545 (error 108)
I20260926 04:07:00.228401 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20706
I20260926 04:07:00.228533 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:00.228377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.129:43545 (local address 127.0.0.1:43658)
0926 04:07:00.228440 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:00.228491 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.129:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 04:07:00.230086 20499 catalog_manager.cc:4584] TS bcd770a3698b4d208ea4b5542919243f (127.0.200.129:43545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f824fbfcd8b4f8bba9559ac2cace87f: Network error: Client connection negotiation failed: client connection to 127.0.200.129:43545: connect: Connection refused (error 111)
I20260926 04:07:00.233775 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20837
I20260926 04:07:00.238418 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 20484
2026-09-26T04:07:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (685 ms)
[----------] 6 tests from EncryptionPolicyTest (8726 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:00 dist-test-slave-j8c1 krb5kdc[21008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:00 dist-test-slave-j8c1 krb5kdc[21008](info): set up 2 sockets
Sep 26 04:07:00 dist-test-slave-j8c1 krb5kdc[21008](info): commencing operation
krb5kdc: starting...
W20260926 04:07:01.278498 802 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:01.401194 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:01.483805 21021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:01.484040 21021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:01.484066 21021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:01.485266 21021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:01.485316 21021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:01.485330 21021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:01.485345 21021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:01.485357 21021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:01.487076 21021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:01.487310 21021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:01.487505 21021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:01.492630 21029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:01.492632 21027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:01.492637 21026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:01.492904 21021 server_base.cc:1034] running on GCE node
I20260926 04:07:01.493221 21021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:01.493485 21021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:01.494659 21021 hybrid_clock.cc:648] HybridClock initialized: now 1790395621494634 us; error 33 us; skew 500 ppm
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:01.497331 21021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:01.497936 21021 webserver.cc:460] Webserver started at http://127.0.0.1:37025/ using document root <none> and password file <none>
I20260926 04:07:01.498193 21021 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:01.498265 21021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:01.498436 21021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:01.499650 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "0bed81699d9d42b7b404e0fc768893c7"
format_stamp: "Formatted at 2026-09-26 04:07:01 on dist-test-slave-j8c1"
server_key: "4dce58ed98d96fa281949fc564890b99"
server_key_iv: "0bbcf430950d1333206b11cb54e051ad"
server_key_version: "encryptionkey@0"
I20260926 04:07:01.500062 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "0bed81699d9d42b7b404e0fc768893c7"
format_stamp: "Formatted at 2026-09-26 04:07:01 on dist-test-slave-j8c1"
server_key: "4dce58ed98d96fa281949fc564890b99"
server_key_iv: "0bbcf430950d1333206b11cb54e051ad"
server_key_version: "encryptionkey@0"
I20260926 04:07:01.501507 21021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:01.502379 21036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:01.502686 21021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:07:01.502800 21021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "0bed81699d9d42b7b404e0fc768893c7"
format_stamp: "Formatted at 2026-09-26 04:07:01 on dist-test-slave-j8c1"
server_key: "4dce58ed98d96fa281949fc564890b99"
server_key_iv: "0bbcf430950d1333206b11cb54e051ad"
server_key_version: "encryptionkey@0"
I20260926 04:07:01.502898 21021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:01.516002 21021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:01.518074 21021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:01.518249 21021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:01.522519 21021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33029
I20260926 04:07:01.522611 21088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33029 every 8 connection(s)
I20260926 04:07:01.522979 21021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:01.523852 21089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:01.526194 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21021
I20260926 04:07:01.526320 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:01.526381 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7: Bootstrap starting.
I20260926 04:07:01.526525 802 external_mini_cluster.cc:1428] Setting key 67e472c7b2f34588abbeb5ef4ea321b3
I20260926 04:07:01.527148 21089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:01.527458 21089 log.cc:824] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:01.528221 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7: No bootstrap required, opened a new log
I20260926 04:07:01.530498 21089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260926 04:07:01.530697 21089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:01.530732 21089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bed81699d9d42b7b404e0fc768893c7, State: Initialized, Role: FOLLOWER
I20260926 04:07:01.530856 21089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260926 04:07:01.530933 21089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:01.530963 21089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:01.531008 21089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:01.531607 21089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260926 04:07:01.531721 21089 leader_election.cc:302] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bed81699d9d42b7b404e0fc768893c7; no voters:
I20260926 04:07:01.531914 21089 leader_election.cc:288] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:01.531980 21094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:01.532124 21094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [term 1 LEADER]: Becoming Leader. State: Replica: 0bed81699d9d42b7b404e0fc768893c7, State: Running, Role: LEADER
I20260926 04:07:01.532199 21089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:01.532223 21094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260926 04:07:01.533486 21096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bed81699d9d42b7b404e0fc768893c7. Latest consensus state: current_term: 1 leader_uuid: "0bed81699d9d42b7b404e0fc768893c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260926 04:07:01.533588 21096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:01.533727 21095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bed81699d9d42b7b404e0fc768893c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bed81699d9d42b7b404e0fc768893c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260926 04:07:01.533954 21095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:01.533806 21110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:01.534368 21110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 04:07:01.535902 21110 catalog_manager.cc:1347] Generated new cluster ID: 9278bbe699fb46919d6e19bb558fa4d1
I20260926 04:07:01.535952 21110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:01.538975 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:01.527114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37345 (local address 127.0.0.1:33029)
0926 04:07:01.527346 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:01.527361 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:01.527386 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:07:01.528040 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:01.528047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:01.528098 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:01.528217 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:01.529033 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:01.529990 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:01.531096 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:01.531526 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:01.531610 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:01.535052 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:01.535056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:01.535057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:01.535069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:01.537165 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:01.537664 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:01.537666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:01.537667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:01.537772 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:01.538022 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:01.538023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:01.538023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:01.538324 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:01.538480 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:01.538674 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 04:07:01.538770 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260926 04:07:01.540136 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:01.548883 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:01.541574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52283 (local address 127.0.0.1:33029)
0926 04:07:01.541615 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:01.541617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:01.541674 (+ 57us) server_negotiation.cc:407] Connection header received
0926 04:07:01.541755 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:01.541756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:01.541766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:01.541791 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:01.542441 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:01.543229 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:01.544208 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:01.544527 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:01.544563 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:01.546018 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:01.546019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:01.546020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:01.546026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:01.547517 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:01.547900 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:01.547901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:01.547901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:01.547947 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:01.548151 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:01.548152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:01.548152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:01.548371 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:01.548474 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:01.548725 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 04:07:01.548791 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:01.556473 21110 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:01.557237 21110 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:01.560612 21110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bed81699d9d42b7b404e0fc768893c7: Generated new TSK 0
I20260926 04:07:01.560802 21110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:01.568094 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21021
W20260926 04:07:01.572646 21006 connection.cc:535] client connection to 127.0.0.1:33029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33029): Connection reset by peer (error 104)
2026-09-26T04:07:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21120](info): set up 2 sockets
Sep 26 04:07:01 dist-test-slave-j8c1 krb5kdc[21120](info): commencing operation
krb5kdc: starting...
W20260926 04:07:02.591212 802 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:02 dist-test-slave-j8c1 krb5kdc[21120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:02.714476 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:02.795764 21133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:02.796037 21133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:02.796077 21133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:02.797295 21133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:02.797376 21133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:02.797403 21133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:02.797435 21133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:02.797457 21133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:02.799098 21133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:02.799387 21133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:02.799636 21133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:02.804726 21139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:02.804855 21133 server_base.cc:1034] running on GCE node
W20260926 04:07:02.804719 21138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:02.804719 21141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:02.805240 21133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:02.805588 21133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:02.806749 21133 hybrid_clock.cc:648] HybridClock initialized: now 1790395622806709 us; error 54 us; skew 500 ppm
Sep 26 04:07:02 dist-test-slave-j8c1 krb5kdc[21120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395622, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:02.814255 21133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:02.814822 21133 webserver.cc:460] Webserver started at http://127.0.0.1:36263/ using document root <none> and password file <none>
I20260926 04:07:02.815081 21133 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:02.815152 21133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:02.815317 21133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:02.816423 21133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5"
format_stamp: "Formatted at 2026-09-26 04:07:02 on dist-test-slave-j8c1"
server_key: "28f986a6358b71ff67fc7d5174ee9dfd"
server_key_iv: "25043c34a1189ad6955951206d7661d2"
server_key_version: "encryptionkey@0"
I20260926 04:07:02.816797 21133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5"
format_stamp: "Formatted at 2026-09-26 04:07:02 on dist-test-slave-j8c1"
server_key: "28f986a6358b71ff67fc7d5174ee9dfd"
server_key_iv: "25043c34a1189ad6955951206d7661d2"
server_key_version: "encryptionkey@0"
I20260926 04:07:02.818244 21133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:02.819223 21148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:02.819484 21133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:07:02.819590 21133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5"
format_stamp: "Formatted at 2026-09-26 04:07:02 on dist-test-slave-j8c1"
server_key: "28f986a6358b71ff67fc7d5174ee9dfd"
server_key_iv: "25043c34a1189ad6955951206d7661d2"
server_key_version: "encryptionkey@0"
I20260926 04:07:02.819667 21133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:02.837051 21133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:02.839066 21133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:02.839222 21133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:02.843041 21133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37241
I20260926 04:07:02.843113 21200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37241 every 8 connection(s)
I20260926 04:07:02.843452 21133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:02.844342 21201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:02.846599 21201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5: Bootstrap starting.
I20260926 04:07:02.847210 21201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:02.847514 21201 log.cc:824] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:02.848487 21201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5: No bootstrap required, opened a new log
I20260926 04:07:02.849611 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21133
I20260926 04:07:02.849727 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:02.849927 802 external_mini_cluster.cc:1428] Setting key 02d3ac8c1fa15bd54dd6577b5ec4b7d7
I20260926 04:07:02.850908 21201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } }
I20260926 04:07:02.851084 21201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:02.851126 21201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbeb607fbb64f1eb7f2faf3cb8415f5, State: Initialized, Role: FOLLOWER
I20260926 04:07:02.851261 21201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } }
I20260926 04:07:02.851342 21201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:02.851364 21201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:02.851432 21201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:02.851963 21201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } }
I20260926 04:07:02.852047 21201 leader_election.cc:302] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bbeb607fbb64f1eb7f2faf3cb8415f5; no voters:
I20260926 04:07:02.852243 21201 leader_election.cc:288] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:02.852272 21206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:02.852381 21206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [term 1 LEADER]: Becoming Leader. State: Replica: 5bbeb607fbb64f1eb7f2faf3cb8415f5, State: Running, Role: LEADER
I20260926 04:07:02.852511 21201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:02.852515 21206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } }
I20260926 04:07:02.852977 21207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } } }
I20260926 04:07:02.853123 21207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:02.853659 21215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:02.854033 21208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bbeb607fbb64f1eb7f2faf3cb8415f5. Latest consensus state: current_term: 1 leader_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbeb607fbb64f1eb7f2faf3cb8415f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37241 } } }
I20260926 04:07:02.854166 21208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:02.854221 21215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:02.855741 21215 catalog_manager.cc:1347] Generated new cluster ID: 1090e6f2c89f4163aa91e4bf2184691e
I20260926 04:07:02.855803 21215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:02.855967 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:02.850500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58321 (local address 127.0.0.1:37241)
0926 04:07:02.850726 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:02.850735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:02.850768 (+ 33us) server_negotiation.cc:407] Connection header received
0926 04:07:02.850861 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:02.850867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:02.850908 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:02.851022 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:02.851254 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:02.852660 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:02.853810 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:02.854468 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:02.854606 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:02.855272 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:02.855275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:02.855275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:02.855490 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:02.855520 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:02.855531 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:02.855623 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:02.855704 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 04:07:02.855732 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":151,"threads_started":1}
I20260926 04:07:02.856899 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:02.861148 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:02.858346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33835 (local address 127.0.0.1:37241)
0926 04:07:02.858387 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:02.858389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:02.858507 (+ 118us) server_negotiation.cc:407] Connection header received
0926 04:07:02.858530 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:02.858531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:02.858539 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:02.858638 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:02.858790 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:02.859238 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:02.859929 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:02.860152 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:02.860228 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:02.860892 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:02.860893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:02.860894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:02.860898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:02.860912 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:02.860916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:02.860969 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:02.861038 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 04:07:02.861051 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:07:02.884248 21215 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:02.884981 21215 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:02.892855 21215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bbeb607fbb64f1eb7f2faf3cb8415f5: Generated new TSK 0
I20260926 04:07:02.893046 21215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:02.895342 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21133
W20260926 04:07:02.899871 21118 connection.cc:535] client connection to 127.0.0.1:37241 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37241): Connection reset by peer (error 104)
2026-09-26T04:07:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:02 dist-test-slave-j8c1 krb5kdc[21232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:02 dist-test-slave-j8c1 krb5kdc[21232](info): set up 2 sockets
Sep 26 04:07:02 dist-test-slave-j8c1 krb5kdc[21232](info): commencing operation
krb5kdc: starting...
W20260926 04:07:03.954658 802 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:04 dist-test-slave-j8c1 krb5kdc[21232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:04.078073 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:04.159242 21245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:04.159516 21245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:04.159564 21245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:04.160756 21245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:04.160836 21245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:04.160861 21245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:04.160892 21245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:04.160970 21245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:04.162488 21245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:04.162788 21245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:04.163022 21245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:04.168054 21250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:04.168090 21251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:04.168171 21245 server_base.cc:1034] running on GCE node
W20260926 04:07:04.168054 21253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:04.168596 21245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:04.168910 21245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:04.170066 21245 hybrid_clock.cc:648] HybridClock initialized: now 1790395624170056 us; error 32 us; skew 500 ppm
Sep 26 04:07:04 dist-test-slave-j8c1 krb5kdc[21232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395624, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:04.172838 21245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:04.173357 21245 webserver.cc:460] Webserver started at http://127.0.0.1:40189/ using document root <none> and password file <none>
I20260926 04:07:04.173614 21245 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:04.173686 21245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:04.173835 21245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:04.175048 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "c9ddd8902fd7483a8533f3bd7e32a06f"
format_stamp: "Formatted at 2026-09-26 04:07:04 on dist-test-slave-j8c1"
server_key: "e63fb3af752e318c9a5133aea61eb33b"
server_key_iv: "0d14d4e5acac98df46ea230496591767"
server_key_version: "encryptionkey@0"
I20260926 04:07:04.175434 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "c9ddd8902fd7483a8533f3bd7e32a06f"
format_stamp: "Formatted at 2026-09-26 04:07:04 on dist-test-slave-j8c1"
server_key: "e63fb3af752e318c9a5133aea61eb33b"
server_key_iv: "0d14d4e5acac98df46ea230496591767"
server_key_version: "encryptionkey@0"
I20260926 04:07:04.176826 21245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 04:07:04.177570 21260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:04.177769 21245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:04.177875 21245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "c9ddd8902fd7483a8533f3bd7e32a06f"
format_stamp: "Formatted at 2026-09-26 04:07:04 on dist-test-slave-j8c1"
server_key: "e63fb3af752e318c9a5133aea61eb33b"
server_key_iv: "0d14d4e5acac98df46ea230496591767"
server_key_version: "encryptionkey@0"
I20260926 04:07:04.177950 21245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:04.181461 21245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:04.183895 21245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:04.184057 21245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:04.188050 21245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45729
I20260926 04:07:04.188081 21312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45729 every 8 connection(s)
I20260926 04:07:04.188429 21245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:04.189419 21313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:04.191737 21313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f: Bootstrap starting.
I20260926 04:07:04.192308 21313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:04.192605 21313 log.cc:824] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:04.192910 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21245
I20260926 04:07:04.193054 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:04.193260 802 external_mini_cluster.cc:1428] Setting key cc1599855f041ba6b07b19848c349911
I20260926 04:07:04.193356 21313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f: No bootstrap required, opened a new log
I20260926 04:07:04.195245 21313 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } }
I20260926 04:07:04.195413 21313 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:04.195456 21313 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ddd8902fd7483a8533f3bd7e32a06f, State: Initialized, Role: FOLLOWER
I20260926 04:07:04.195643 21313 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } }
I20260926 04:07:04.195770 21313 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:04.195816 21313 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:04.195870 21313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:04.196588 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:04.193811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48565 (local address 127.0.0.1:45729)
0926 04:07:04.193983 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:04.193990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:04.194016 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:07:04.195228 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:04.195234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:04.195262 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:04.195379 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:04.196094 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:04.196097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:04.196098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:04.196101 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:04.196138 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:04.196149 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:04.196203 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:04.196292 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 04:07:04.196342 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260926 04:07:04.197001 21313 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } }
I20260926 04:07:04.197126 21313 leader_election.cc:302] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ddd8902fd7483a8533f3bd7e32a06f; no voters:
I20260926 04:07:04.197283 21313 leader_election.cc:288] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:04.197392 21318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:04.197618 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:04.197579 21318 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [term 1 LEADER]: Becoming Leader. State: Replica: c9ddd8902fd7483a8533f3bd7e32a06f, State: Running, Role: LEADER
I20260926 04:07:04.197561 21313 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:04.197717 21318 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } }
I20260926 04:07:04.198019 21318 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ddd8902fd7483a8533f3bd7e32a06f. Latest consensus state: current_term: 1 leader_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } } }
I20260926 04:07:04.198149 21318 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:04.198642 21326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:04.199177 21326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:04.199268 21319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ddd8902fd7483a8533f3bd7e32a06f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45729 } } }
I20260926 04:07:04.199378 21319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:04.200858 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:04.199607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37985 (local address 127.0.0.1:45729)
0926 04:07:04.199638 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:04.199640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:04.199861 (+ 221us) server_negotiation.cc:407] Connection header received
0926 04:07:04.199932 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:04.199933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:04.199939 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:04.199983 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:04.200628 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:04.200629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:04.200629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:04.200634 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:04.200650 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:04.200654 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:04.200722 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:04.200782 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 04:07:04.200797 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260926 04:07:04.202670 21326 catalog_manager.cc:1347] Generated new cluster ID: c5eca7054b8447cf84329f1e27b647ae
I20260926 04:07:04.202724 21326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:04.216917 21326 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:04.217123 21326 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:04.228785 21326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ddd8902fd7483a8533f3bd7e32a06f: Generated new TSK 0
I20260926 04:07:04.229043 21326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:04.238492 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21245
2026-09-26T04:07:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:04 dist-test-slave-j8c1 krb5kdc[21344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:04 dist-test-slave-j8c1 krb5kdc[21344](info): set up 2 sockets
Sep 26 04:07:04 dist-test-slave-j8c1 krb5kdc[21344](info): commencing operation
krb5kdc: starting...
W20260926 04:07:05.282774 802 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:05.410041 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:05.490917 21357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:05.491185 21357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:05.491226 21357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:05.492434 21357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:05.492520 21357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:05.492544 21357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:05.492568 21357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:05.492596 21357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:05.494163 21357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:05.494455 21357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:05.494728 21357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:05.499725 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:05.499795 21357 server_base.cc:1034] running on GCE node
W20260926 04:07:05.499758 21363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:05.499725 21365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:05.500391 21357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:05.500691 21357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:05.501863 21357 hybrid_clock.cc:648] HybridClock initialized: now 1790395625501852 us; error 45 us; skew 500 ppm
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:05.504406 21357 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:05.504897 21357 webserver.cc:460] Webserver started at http://127.0.0.1:45217/ using document root <none> and password file <none>
I20260926 04:07:05.505137 21357 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:05.505209 21357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:05.505364 21357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:05.506443 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "5f6f9b2cccff41ee8a59c699009ac1bc"
format_stamp: "Formatted at 2026-09-26 04:07:05 on dist-test-slave-j8c1"
server_key: "8e23288a4bdafc7bd5ea53931e2374f1"
server_key_iv: "4912f647451bd31600eda66e98996312"
server_key_version: "encryptionkey@0"
I20260926 04:07:05.506901 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "5f6f9b2cccff41ee8a59c699009ac1bc"
format_stamp: "Formatted at 2026-09-26 04:07:05 on dist-test-slave-j8c1"
server_key: "8e23288a4bdafc7bd5ea53931e2374f1"
server_key_iv: "4912f647451bd31600eda66e98996312"
server_key_version: "encryptionkey@0"
I20260926 04:07:05.508381 21357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:05.509172 21372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:05.509363 21357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:05.509465 21357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "5f6f9b2cccff41ee8a59c699009ac1bc"
format_stamp: "Formatted at 2026-09-26 04:07:05 on dist-test-slave-j8c1"
server_key: "8e23288a4bdafc7bd5ea53931e2374f1"
server_key_iv: "4912f647451bd31600eda66e98996312"
server_key_version: "encryptionkey@0"
I20260926 04:07:05.509564 21357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:05.523373 21357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:05.526041 21357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:05.526219 21357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:05.530135 21357 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40431
I20260926 04:07:05.530216 21424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40431 every 8 connection(s)
I20260926 04:07:05.530467 21357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:05.531435 21425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:05.533880 21425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: Bootstrap starting.
I20260926 04:07:05.534511 21425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:05.534893 21425 log.cc:824] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:05.535336 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21357
I20260926 04:07:05.535494 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:05.535718 802 external_mini_cluster.cc:1428] Setting key a40902a061f0d651ffc079b934095edb
I20260926 04:07:05.535842 21425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: No bootstrap required, opened a new log
I20260926 04:07:05.538331 21425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260926 04:07:05.538537 21425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:05.538615 21425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6f9b2cccff41ee8a59c699009ac1bc, State: Initialized, Role: FOLLOWER
I20260926 04:07:05.538790 21425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260926 04:07:05.538875 21425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:05.538908 21425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:05.538959 21425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:05.539743 21425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260926 04:07:05.539860 21425 leader_election.cc:302] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f6f9b2cccff41ee8a59c699009ac1bc; no voters:
I20260926 04:07:05.540030 21425 leader_election.cc:288] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:05.540087 21430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:05.540267 21430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [term 1 LEADER]: Becoming Leader. State: Replica: 5f6f9b2cccff41ee8a59c699009ac1bc, State: Running, Role: LEADER
I20260926 04:07:05.540299 21425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:05.540359 21430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } }
I20260926 04:07:05.541673 21432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f6f9b2cccff41ee8a59c699009ac1bc. Latest consensus state: current_term: 1 leader_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } } }
I20260926 04:07:05.541980 21431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f6f9b2cccff41ee8a59c699009ac1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40431 } } }
I20260926 04:07:05.542037 21431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:05.541802 21432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc [sys.catalog]: This master's current role is: LEADER
W20260926 04:07:05.542366 21445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:05.542429 21445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:05.542518 21446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:05.542752 21446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:05.544560 21446 catalog_manager.cc:1347] Generated new cluster ID: 07ac7dff01334a8abab7d6f15d1775ed
I20260926 04:07:05.544620 21446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 04:07:05.549361 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:05.536289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34579 (local address 127.0.0.1:40431)
0926 04:07:05.536503 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:05.536513 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:05.536552 (+ 39us) server_negotiation.cc:407] Connection header received
0926 04:07:05.537421 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:05.537432 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:05.537501 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:05.537671 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:05.538271 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:05.540172 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:05.541393 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:05.542130 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:05.542260 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:05.545481 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:05.545486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:05.545486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:05.545499 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:05.547543 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:05.547987 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:05.547991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:05.547991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:05.548095 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:05.548332 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:05.548333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:05.548334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:05.548611 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:05.548808 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:05.548917 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 04:07:05.549188 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260926 04:07:05.550407 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:05.558974 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:05.551985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59447 (local address 127.0.0.1:40431)
0926 04:07:05.552025 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:05.552027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:05.552157 (+ 130us) server_negotiation.cc:407] Connection header received
0926 04:07:05.552243 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:05.552244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:05.552254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:05.552303 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:05.552874 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:05.553432 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:05.554288 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:05.554540 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:05.554656 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:05.555981 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:05.555983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:05.555983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:05.555990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:05.557697 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:05.558078 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:05.558079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:05.558079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:05.558148 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:05.558370 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:05.558371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:05.558371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:05.558502 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:05.558653 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:05.558765 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 04:07:05.558852 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:05.562692 21446 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:05.563429 21446 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:05.575819 21446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f6f9b2cccff41ee8a59c699009ac1bc: Generated new TSK 0
I20260926 04:07:05.576074 21446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:05.587670 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21357
2026-09-26T04:07:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21456](info): set up 2 sockets
Sep 26 04:07:05 dist-test-slave-j8c1 krb5kdc[21456](info): commencing operation
krb5kdc: starting...
W20260926 04:07:06.639148 802 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:06 dist-test-slave-j8c1 krb5kdc[21456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:06.766237 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:06.848596 21469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:06.848804 21469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:06.848826 21469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:06.849915 21469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:06.849954 21469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:06.849968 21469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:06.849982 21469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:06.849993 21469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:06.851498 21469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:06.851696 21469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:06.851862 21469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:06.856565 21477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:06.856566 21475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:06.856626 21474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:06.856789 21469 server_base.cc:1034] running on GCE node
I20260926 04:07:06.857033 21469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:06.857311 21469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:06.858476 21469 hybrid_clock.cc:648] HybridClock initialized: now 1790395626858463 us; error 67 us; skew 500 ppm
Sep 26 04:07:06 dist-test-slave-j8c1 krb5kdc[21456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:06.860929 21469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:06.861402 21469 webserver.cc:460] Webserver started at http://127.0.0.1:34661/ using document root <none> and password file <none>
I20260926 04:07:06.861647 21469 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:06.861718 21469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:06.861869 21469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:06.862978 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "7cb00ccc6c9942faa4e5647cbca5888c"
format_stamp: "Formatted at 2026-09-26 04:07:06 on dist-test-slave-j8c1"
server_key: "16feef7e7b7908e5cab33f016307314e"
server_key_iv: "3f0ff5ef6824b178fbbfef94901249a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:06.863327 21469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "7cb00ccc6c9942faa4e5647cbca5888c"
format_stamp: "Formatted at 2026-09-26 04:07:06 on dist-test-slave-j8c1"
server_key: "16feef7e7b7908e5cab33f016307314e"
server_key_iv: "3f0ff5ef6824b178fbbfef94901249a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:06.864673 21469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:07:06.865419 21484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:06.865634 21469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:06.865743 21469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "7cb00ccc6c9942faa4e5647cbca5888c"
format_stamp: "Formatted at 2026-09-26 04:07:06 on dist-test-slave-j8c1"
server_key: "16feef7e7b7908e5cab33f016307314e"
server_key_iv: "3f0ff5ef6824b178fbbfef94901249a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:06.865818 21469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:06.878248 21469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:06.880236 21469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:06.880401 21469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:06.884418 21469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42141
I20260926 04:07:06.884452 21536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42141 every 8 connection(s)
I20260926 04:07:06.884833 21469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:06.885803 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:06.888190 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c: Bootstrap starting.
I20260926 04:07:06.888779 21537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:06.889070 21537 log.cc:824] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:06.889788 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c: No bootstrap required, opened a new log
I20260926 04:07:06.891327 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21469
I20260926 04:07:06.891458 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:06.891466 21537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } }
I20260926 04:07:06.891660 21537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:06.891688 21537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb00ccc6c9942faa4e5647cbca5888c, State: Initialized, Role: FOLLOWER
I20260926 04:07:06.891726 802 external_mini_cluster.cc:1428] Setting key 3cd4c554515322cfe099152b492d1b64
I20260926 04:07:06.891887 21537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } }
I20260926 04:07:06.891986 21537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:06.892030 21537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:06.892087 21537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:06.893250 21537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } }
I20260926 04:07:06.893361 21537 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb00ccc6c9942faa4e5647cbca5888c; no voters:
I20260926 04:07:06.893510 21537 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:06.893589 21542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:06.893772 21537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:06.893793 21542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [term 1 LEADER]: Becoming Leader. State: Replica: 7cb00ccc6c9942faa4e5647cbca5888c, State: Running, Role: LEADER
I20260926 04:07:06.893882 21542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } }
I20260926 04:07:06.894268 21543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } } }
I20260926 04:07:06.894372 21543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:06.894256 21542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb00ccc6c9942faa4e5647cbca5888c. Latest consensus state: current_term: 1 leader_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb00ccc6c9942faa4e5647cbca5888c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42141 } } }
I20260926 04:07:06.894591 21542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:06.894730 21551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:06.895215 21551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:06.897110 21551 catalog_manager.cc:1347] Generated new cluster ID: 2ee779c10a104d7697291607dc35942a
I20260926 04:07:06.897167 21551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:06.897850 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:06.892241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54195 (local address 127.0.0.1:42141)
0926 04:07:06.892457 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:06.892467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:06.892503 (+ 36us) server_negotiation.cc:407] Connection header received
0926 04:07:06.892630 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:06.892640 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:06.892698 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:06.892894 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:06.893031 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:06.894440 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:06.895722 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:06.896226 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:06.896336 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:06.897079 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:06.897082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:06.897082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:06.897360 (+ 278us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:06.897392 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:06.897404 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:06.897487 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:06.897565 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 04:07:06.897635 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260926 04:07:06.898774 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:06.905767 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:06.902904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42347 (local address 127.0.0.1:42141)
0926 04:07:06.902942 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:06.902945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:06.903062 (+ 117us) server_negotiation.cc:407] Connection header received
0926 04:07:06.903123 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:06.903124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:06.903134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:06.903179 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:06.903309 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:06.903791 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:06.904514 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:06.904719 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:06.904798 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:06.905488 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:06.905490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:06.905490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:06.905495 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:06.905510 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:06.905514 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:06.905564 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:06.905627 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 04:07:06.905654 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260926 04:07:06.911962 21551 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:06.912580 21551 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:06.917095 21551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb00ccc6c9942faa4e5647cbca5888c: Generated new TSK 0
I20260926 04:07:06.917325 21551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:06.923025 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21469
2026-09-26T04:07:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:06 dist-test-slave-j8c1 krb5kdc[21568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:06 dist-test-slave-j8c1 krb5kdc[21568](info): set up 2 sockets
Sep 26 04:07:06 dist-test-slave-j8c1 krb5kdc[21568](info): commencing operation
krb5kdc: starting...
W20260926 04:07:07.962770 802 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:08 dist-test-slave-j8c1 krb5kdc[21568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:08.087179 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:08.171341 21581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:08.171571 21581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:08.171597 21581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:08.172746 21581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:08.172786 21581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:08.172801 21581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:08.172814 21581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:08.172827 21581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:08.174296 21581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:08.174506 21581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:08.174791 21581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:08.180622 21586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:08.180629 21587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:08.180877 21581 server_base.cc:1034] running on GCE node
W20260926 04:07:08.180634 21589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:08.181165 21581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:08.181468 21581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:08.182649 21581 hybrid_clock.cc:648] HybridClock initialized: now 1790395628182643 us; error 40 us; skew 500 ppm
Sep 26 04:07:08 dist-test-slave-j8c1 krb5kdc[21568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:08.189874 21581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:08.190531 21581 webserver.cc:460] Webserver started at http://127.0.0.1:45501/ using document root <none> and password file <none>
I20260926 04:07:08.190807 21581 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:08.190850 21581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:08.190946 21581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:08.191983 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "284cd6a6d3f34e19a4da0f95a12baaf8"
format_stamp: "Formatted at 2026-09-26 04:07:08 on dist-test-slave-j8c1"
server_key: "88a86fba8db02e025446aae99cb3933d"
server_key_iv: "a65e16afe173650055cec40271189416"
server_key_version: "encryptionkey@0"
I20260926 04:07:08.192302 21581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "284cd6a6d3f34e19a4da0f95a12baaf8"
format_stamp: "Formatted at 2026-09-26 04:07:08 on dist-test-slave-j8c1"
server_key: "88a86fba8db02e025446aae99cb3933d"
server_key_iv: "a65e16afe173650055cec40271189416"
server_key_version: "encryptionkey@0"
I20260926 04:07:08.193717 21581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:08.194680 21596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:08.194895 21581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:08.195000 21581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "284cd6a6d3f34e19a4da0f95a12baaf8"
format_stamp: "Formatted at 2026-09-26 04:07:08 on dist-test-slave-j8c1"
server_key: "88a86fba8db02e025446aae99cb3933d"
server_key_iv: "a65e16afe173650055cec40271189416"
server_key_version: "encryptionkey@0"
I20260926 04:07:08.195080 21581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:08.198454 21581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:08.200764 21581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:08.200929 21581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:08.206144 21581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34173
I20260926 04:07:08.206174 21648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34173 every 8 connection(s)
I20260926 04:07:08.206598 21581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:08.207681 21649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:08.210698 21649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: Bootstrap starting.
I20260926 04:07:08.211269 21649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:08.211521 21649 log.cc:824] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:08.212229 21649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: No bootstrap required, opened a new log
I20260926 04:07:08.212971 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21581
I20260926 04:07:08.213088 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:08.213282 802 external_mini_cluster.cc:1428] Setting key a2824590a79a04287e6c80c3b699b917
I20260926 04:07:08.214452 21649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } }
I20260926 04:07:08.214682 21649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:08.214726 21649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284cd6a6d3f34e19a4da0f95a12baaf8, State: Initialized, Role: FOLLOWER
I20260926 04:07:08.214905 21649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } }
I20260926 04:07:08.215040 21649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:08.215101 21649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:08.215157 21649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:08.216105 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:08.213837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41287 (local address 127.0.0.1:34173)
0926 04:07:08.214028 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:08.214036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:08.214062 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:07:08.214426 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:08.214432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:08.214459 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:08.214641 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:08.215379 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:08.215383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:08.215383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:08.215388 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:08.215438 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:08.215453 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:08.215532 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:08.215772 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 04:07:08.215825 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260926 04:07:08.216511 21649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } }
I20260926 04:07:08.216619 21649 leader_election.cc:302] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284cd6a6d3f34e19a4da0f95a12baaf8; no voters:
I20260926 04:07:08.216779 21649 leader_election.cc:288] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:08.216804 21654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:08.216919 21654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [term 1 LEADER]: Becoming Leader. State: Replica: 284cd6a6d3f34e19a4da0f95a12baaf8, State: Running, Role: LEADER
I20260926 04:07:08.217046 21649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:08.217021 21654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } }
I20260926 04:07:08.217228 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:08.217453 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284cd6a6d3f34e19a4da0f95a12baaf8. Latest consensus state: current_term: 1 leader_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } } }
I20260926 04:07:08.217525 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:08.218791 21656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284cd6a6d3f34e19a4da0f95a12baaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34173 } } }
I20260926 04:07:08.218878 21656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8 [sys.catalog]: This master's current role is: LEADER
W20260926 04:07:08.218899 21672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:08.218955 21672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:08.219020 21660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:08.219170 21660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:08.220646 21660 catalog_manager.cc:1347] Generated new cluster ID: 60c4f284c36648528fc4db28d6403151
I20260926 04:07:08.220692 21660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:08.224206 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:08.222945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33997 (local address 127.0.0.1:34173)
0926 04:07:08.222985 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:08.222987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:08.223082 (+ 95us) server_negotiation.cc:407] Connection header received
0926 04:07:08.223140 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:08.223141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:08.223146 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:08.223217 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:08.223971 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:08.223973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:08.223973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:08.223977 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:08.223991 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:08.223996 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:08.224050 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:08.224123 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 04:07:08.224136 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:08.244926 21660 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:08.245126 21660 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:08.257512 21660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284cd6a6d3f34e19a4da0f95a12baaf8: Generated new TSK 0
I20260926 04:07:08.257766 21660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:08.265990 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21581
2026-09-26T04:07:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:08 dist-test-slave-j8c1 krb5kdc[21680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:08 dist-test-slave-j8c1 krb5kdc[21680](info): set up 2 sockets
Sep 26 04:07:08 dist-test-slave-j8c1 krb5kdc[21680](info): commencing operation
krb5kdc: starting...
W20260926 04:07:09.323246 802 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:09.453701 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:09.537421 21693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:09.537658 21693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:09.537684 21693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:09.539034 21693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:09.539085 21693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:09.539100 21693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:09.539113 21693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:09.539126 21693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:09.540745 21693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:09.541024 21693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:09.541288 21693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:09.546151 21701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:09.546135 21699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:09.546222 21698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:09.546716 21693 server_base.cc:1034] running on GCE node
I20260926 04:07:09.546980 21693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:09.547235 21693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:09.548391 21693 hybrid_clock.cc:648] HybridClock initialized: now 1790395629548378 us; error 29 us; skew 500 ppm
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:09.551113 21693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:09.551581 21693 webserver.cc:460] Webserver started at http://127.0.0.1:40379/ using document root <none> and password file <none>
I20260926 04:07:09.551816 21693 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:09.551864 21693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:09.551987 21693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:09.553104 21693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "3096af2dfe7f4d75a64361ab6ac4e817"
format_stamp: "Formatted at 2026-09-26 04:07:09 on dist-test-slave-j8c1"
server_key: "0fefe165349188586f926e33414d123b"
server_key_iv: "954082ceb66700fb9d82450b1112cd30"
server_key_version: "encryptionkey@0"
I20260926 04:07:09.553448 21693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "3096af2dfe7f4d75a64361ab6ac4e817"
format_stamp: "Formatted at 2026-09-26 04:07:09 on dist-test-slave-j8c1"
server_key: "0fefe165349188586f926e33414d123b"
server_key_iv: "954082ceb66700fb9d82450b1112cd30"
server_key_version: "encryptionkey@0"
I20260926 04:07:09.554860 21693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:07:09.555676 21708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:09.555932 21693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:09.556062 21693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "3096af2dfe7f4d75a64361ab6ac4e817"
format_stamp: "Formatted at 2026-09-26 04:07:09 on dist-test-slave-j8c1"
server_key: "0fefe165349188586f926e33414d123b"
server_key_iv: "954082ceb66700fb9d82450b1112cd30"
server_key_version: "encryptionkey@0"
I20260926 04:07:09.556143 21693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:09.576249 21693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:09.578712 21693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:09.578840 21693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:09.583004 21693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33187
I20260926 04:07:09.583478 21693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:09.583555 21760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33187 every 8 connection(s)
I20260926 04:07:09.584218 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:09.586637 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: Bootstrap starting.
I20260926 04:07:09.587558 21761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:09.587900 21761 log.cc:824] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:09.588721 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: No bootstrap required, opened a new log
I20260926 04:07:09.589370 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21693
I20260926 04:07:09.589514 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:09.589712 802 external_mini_cluster.cc:1428] Setting key 25c5cb4f1ebba27245b844196b673811
I20260926 04:07:09.591154 21761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260926 04:07:09.591342 21761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:09.591385 21761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3096af2dfe7f4d75a64361ab6ac4e817, State: Initialized, Role: FOLLOWER
I20260926 04:07:09.591560 21761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260926 04:07:09.591660 21761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:09.591694 21761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:09.591747 21761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:09.592541 21761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260926 04:07:09.592669 21761 leader_election.cc:302] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3096af2dfe7f4d75a64361ab6ac4e817; no voters:
I20260926 04:07:09.592880 21761 leader_election.cc:288] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:09.592911 21766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:09.593031 21766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [term 1 LEADER]: Becoming Leader. State: Replica: 3096af2dfe7f4d75a64361ab6ac4e817, State: Running, Role: LEADER
I20260926 04:07:09.593164 21761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:09.593169 21766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260926 04:07:09.593714 21767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } } }
I20260926 04:07:09.593732 21768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3096af2dfe7f4d75a64361ab6ac4e817. Latest consensus state: current_term: 1 leader_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3096af2dfe7f4d75a64361ab6ac4e817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } } }
I20260926 04:07:09.593888 21767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:09.593989 21768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817 [sys.catalog]: This master's current role is: LEADER
W20260926 04:07:09.595139 21782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:09.595197 21782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:09.595252 21775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:09.595455 21775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:09.597460 21775 catalog_manager.cc:1347] Generated new cluster ID: 7e5562aa47b4490db98134e47e20ef59
I20260926 04:07:09.597548 21775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 04:07:09.602046 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:09.590291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:57773 (local address 127.0.0.1:33187)
0926 04:07:09.590485 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:09.590492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:09.590518 (+ 26us) server_negotiation.cc:407] Connection header received
0926 04:07:09.591273 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:09.591280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:09.591336 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:09.591525 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:09.592333 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:09.593758 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:09.594934 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:09.595459 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:09.595598 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:09.598528 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:09.598532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:09.598532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:09.598579 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:09.600330 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:09.600747 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:09.600750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:09.600750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:09.600843 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:09.601212 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:09.601213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:09.601214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:09.601466 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:09.601674 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:09.601773 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 04:07:09.601879 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260926 04:07:09.603129 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:09.614159 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:09.604743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41065 (local address 127.0.0.1:33187)
0926 04:07:09.604793 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:09.604795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:09.606844 (+ 2049us) server_negotiation.cc:407] Connection header received
0926 04:07:09.606936 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:09.606937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:09.606949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:09.607023 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:09.607607 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:09.608128 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:09.609185 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:09.609504 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:09.609607 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:09.611215 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:09.611217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:09.611218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:09.611227 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:09.612839 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:09.613260 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:09.613262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:09.613263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:09.613331 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:09.613569 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:09.613570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:09.613570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:09.613708 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:09.613839 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:09.613956 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 04:07:09.614031 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:07:09.617611 21775 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:09.618356 21775 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:09.643962 21775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3096af2dfe7f4d75a64361ab6ac4e817: Generated new TSK 0
I20260926 04:07:09.644193 21775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:09.654351 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21693
W20260926 04:07:09.658776 21678 connection.cc:535] client connection to 127.0.0.1:33187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33187): Connection reset by peer (error 104)
2026-09-26T04:07:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21792](info): set up 2 sockets
Sep 26 04:07:09 dist-test-slave-j8c1 krb5kdc[21792](info): commencing operation
krb5kdc: starting...
W20260926 04:07:10.702725 802 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:10 dist-test-slave-j8c1 krb5kdc[21792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:10.821969 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:10.901690 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:10.901911 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:10.901934 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:10.903162 21805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:10.903209 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:10.903224 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:10.903237 21805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:10.903249 21805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:10.904749 21805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:10.905000 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:10.905236 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:10.910409 21813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:10.910404 21811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:10.910446 21805 server_base.cc:1034] running on GCE node
W20260926 04:07:10.910404 21810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:10.910943 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:10.911254 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:10.912411 21805 hybrid_clock.cc:648] HybridClock initialized: now 1790395630912395 us; error 30 us; skew 500 ppm
Sep 26 04:07:10 dist-test-slave-j8c1 krb5kdc[21792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:10.914997 21805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:10.915462 21805 webserver.cc:460] Webserver started at http://127.0.0.1:44947/ using document root <none> and password file <none>
I20260926 04:07:10.915695 21805 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:10.915741 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:10.915887 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:10.917064 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "73198e5749ac434aaf5e496b7926617f"
format_stamp: "Formatted at 2026-09-26 04:07:10 on dist-test-slave-j8c1"
server_key: "11328bc205af76abfac4a3d256feeaf1"
server_key_iv: "f4801a1e5f82d8cb816422bde60c6d4c"
server_key_version: "encryptionkey@0"
I20260926 04:07:10.917428 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "73198e5749ac434aaf5e496b7926617f"
format_stamp: "Formatted at 2026-09-26 04:07:10 on dist-test-slave-j8c1"
server_key: "11328bc205af76abfac4a3d256feeaf1"
server_key_iv: "f4801a1e5f82d8cb816422bde60c6d4c"
server_key_version: "encryptionkey@0"
I20260926 04:07:10.918879 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:10.919752 21820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:10.920012 21805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:10.920142 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "73198e5749ac434aaf5e496b7926617f"
format_stamp: "Formatted at 2026-09-26 04:07:10 on dist-test-slave-j8c1"
server_key: "11328bc205af76abfac4a3d256feeaf1"
server_key_iv: "f4801a1e5f82d8cb816422bde60c6d4c"
server_key_version: "encryptionkey@0"
I20260926 04:07:10.920233 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:10.931495 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:10.934036 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:10.934232 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:10.938733 21805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40571
I20260926 04:07:10.938786 21872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40571 every 8 connection(s)
I20260926 04:07:10.939195 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:10.940104 21873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:10.942410 21873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f: Bootstrap starting.
I20260926 04:07:10.943043 21873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:10.943336 21873 log.cc:824] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:10.944073 21873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f: No bootstrap required, opened a new log
I20260926 04:07:10.945652 21873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } }
I20260926 04:07:10.945792 21873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:10.945815 21873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73198e5749ac434aaf5e496b7926617f, State: Initialized, Role: FOLLOWER
I20260926 04:07:10.945992 21873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } }
I20260926 04:07:10.946071 21873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:10.946116 21873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:10.946167 21873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:10.946816 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21805
I20260926 04:07:10.946817 21873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } }
I20260926 04:07:10.946964 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:10.946942 21873 leader_election.cc:302] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73198e5749ac434aaf5e496b7926617f; no voters:
I20260926 04:07:10.947152 802 external_mini_cluster.cc:1428] Setting key 3b18a1e82f855c81d0ee89f87cd4c0db
I20260926 04:07:10.947170 21873 leader_election.cc:288] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:10.947201 21876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:10.947381 21876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [term 1 LEADER]: Becoming Leader. State: Replica: 73198e5749ac434aaf5e496b7926617f, State: Running, Role: LEADER
I20260926 04:07:10.947443 21873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:10.947511 21876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } }
I20260926 04:07:10.948203 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73198e5749ac434aaf5e496b7926617f. Latest consensus state: current_term: 1 leader_uuid: "73198e5749ac434aaf5e496b7926617f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } } }
I20260926 04:07:10.948293 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:10.948622 21877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73198e5749ac434aaf5e496b7926617f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73198e5749ac434aaf5e496b7926617f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40571 } } }
I20260926 04:07:10.948700 21877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:10.948760 21887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:10.949349 21887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:10.950970 21887 catalog_manager.cc:1347] Generated new cluster ID: eeb39ffe376541b694907f9419fa3320
I20260926 04:07:10.951030 21887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:10.953244 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:10.947780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:55753 (local address 127.0.0.1:40571)
0926 04:07:10.947974 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:10.947983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:10.948015 (+ 32us) server_negotiation.cc:407] Connection header received
0926 04:07:10.948144 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:10.948152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:10.948214 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:10.948606 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:10.948727 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:10.950140 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:10.951271 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:10.951699 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:10.951813 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:10.952578 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:10.952581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:10.952582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:10.952782 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:10.952813 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:10.952825 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:10.952918 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:10.952998 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 04:07:10.953028 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260926 04:07:10.954247 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:10.961607 21880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:10.955692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:35463 (local address 127.0.0.1:40571)
0926 04:07:10.955731 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:10.955733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:10.958758 (+ 3025us) server_negotiation.cc:407] Connection header received
0926 04:07:10.958825 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:10.958826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:10.958839 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:10.958895 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:10.959041 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:10.959603 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:10.960318 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:10.960552 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:10.960652 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:10.961345 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:10.961346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:10.961346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:10.961351 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:10.961366 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:10.961370 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:10.961424 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:10.961482 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 04:07:10.961494 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:07:10.964178 21887 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:10.964931 21887 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:10.969630 21887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73198e5749ac434aaf5e496b7926617f: Generated new TSK 0
I20260926 04:07:10.969833 21887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:10.976883 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21805
W20260926 04:07:10.981575 21790 connection.cc:535] client connection to 127.0.0.1:40571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40571): Connection reset by peer (error 104)
2026-09-26T04:07:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:11 dist-test-slave-j8c1 krb5kdc[21904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:11 dist-test-slave-j8c1 krb5kdc[21904](info): set up 2 sockets
Sep 26 04:07:11 dist-test-slave-j8c1 krb5kdc[21904](info): commencing operation
krb5kdc: starting...
W20260926 04:07:12.015045 802 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:12 dist-test-slave-j8c1 krb5kdc[21904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:12.138904 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:12.221246 21917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:12.221473 21917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:12.221498 21917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:12.222622 21917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:12.222658 21917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:12.222672 21917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:12.222685 21917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:12.222698 21917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:12.224099 21917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:12.224300 21917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:12.224480 21917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:12.229848 21925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:12.229986 21917 server_base.cc:1034] running on GCE node
W20260926 04:07:12.229868 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:12.230111 21922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:12.230396 21917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:12.230718 21917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:12.231873 21917 hybrid_clock.cc:648] HybridClock initialized: now 1790395632231868 us; error 34 us; skew 500 ppm
Sep 26 04:07:12 dist-test-slave-j8c1 krb5kdc[21904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:12.234444 21917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:12.234982 21917 webserver.cc:460] Webserver started at http://127.0.0.1:42685/ using document root <none> and password file <none>
I20260926 04:07:12.235211 21917 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:12.235286 21917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:12.235448 21917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:12.236542 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "84dbb44856504f5f82a6acd52798a721"
format_stamp: "Formatted at 2026-09-26 04:07:12 on dist-test-slave-j8c1"
server_key: "470bd97c932101d30f5a02e70396073b"
server_key_iv: "81b9856b06eba269289ca82747b5d799"
server_key_version: "encryptionkey@0"
I20260926 04:07:12.236910 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "84dbb44856504f5f82a6acd52798a721"
format_stamp: "Formatted at 2026-09-26 04:07:12 on dist-test-slave-j8c1"
server_key: "470bd97c932101d30f5a02e70396073b"
server_key_iv: "81b9856b06eba269289ca82747b5d799"
server_key_version: "encryptionkey@0"
I20260926 04:07:12.238317 21917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:12.239225 21932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:12.239475 21917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:12.239598 21917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "84dbb44856504f5f82a6acd52798a721"
format_stamp: "Formatted at 2026-09-26 04:07:12 on dist-test-slave-j8c1"
server_key: "470bd97c932101d30f5a02e70396073b"
server_key_iv: "81b9856b06eba269289ca82747b5d799"
server_key_version: "encryptionkey@0"
I20260926 04:07:12.239689 21917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:12.243294 21917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:12.245731 21917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:12.245905 21917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:12.250008 21917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39889
I20260926 04:07:12.250043 21984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39889 every 8 connection(s)
I20260926 04:07:12.250396 21917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:12.251494 21985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:12.253796 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 21917
I20260926 04:07:12.253929 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:12.254040 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721: Bootstrap starting.
I20260926 04:07:12.254151 802 external_mini_cluster.cc:1428] Setting key 6d21f356b90b2bf9257028cd29bc2d11
I20260926 04:07:12.254851 21985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:12.255172 21985 log.cc:824] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:12.256062 21985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721: No bootstrap required, opened a new log
I20260926 04:07:12.256860 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:12.254709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:38019 (local address 127.0.0.1:39889)
0926 04:07:12.254928 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:12.254937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:12.254971 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:07:12.255547 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:12.255556 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:12.255591 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:12.255745 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:12.256448 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:12.256452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:12.256452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:12.256456 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:12.256506 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:12.256517 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:12.256609 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:12.256647 (+ 38us) server_negotiation.cc:299] Negotiation successful
0926 04:07:12.256690 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260926 04:07:12.257862 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:12.258724 21985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260926 04:07:12.258886 21985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:12.258934 21985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84dbb44856504f5f82a6acd52798a721, State: Initialized, Role: FOLLOWER
I20260926 04:07:12.259115 21985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260926 04:07:12.259249 21985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:12.259296 21985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:12.259347 21985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:12.259917 21985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260926 04:07:12.260037 21985 leader_election.cc:302] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84dbb44856504f5f82a6acd52798a721; no voters:
I20260926 04:07:12.260221 21985 leader_election.cc:288] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:12.260267 21994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:12.260442 21994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [term 1 LEADER]: Becoming Leader. State: Replica: 84dbb44856504f5f82a6acd52798a721, State: Running, Role: LEADER
I20260926 04:07:12.260476 21985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:12.260528 21994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } }
I20260926 04:07:12.260952 21996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84dbb44856504f5f82a6acd52798a721. Latest consensus state: current_term: 1 leader_uuid: "84dbb44856504f5f82a6acd52798a721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260926 04:07:12.261068 21996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:12.261292 22003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:12.260941 21995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84dbb44856504f5f82a6acd52798a721" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84dbb44856504f5f82a6acd52798a721" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39889 } } }
I20260926 04:07:12.261507 21995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:12.261780 22003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:12.263515 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:12.261960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41171 (local address 127.0.0.1:39889)
0926 04:07:12.261999 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:12.262001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:12.262350 (+ 349us) server_negotiation.cc:407] Connection header received
0926 04:07:12.262396 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:12.262397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:12.262403 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:12.262479 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:12.263283 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:12.263285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:12.263285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:12.263290 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:12.263307 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:12.263311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:12.263377 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:12.263442 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 04:07:12.263456 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:12.263689 22003 catalog_manager.cc:1347] Generated new cluster ID: c9eaa91004f84c28b5e8da0140d47c8f
I20260926 04:07:12.263756 22003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:12.278719 22003 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:12.278934 22003 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:12.282184 22003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84dbb44856504f5f82a6acd52798a721: Generated new TSK 0
I20260926 04:07:12.282393 22003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:12.290617 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 21917
2026-09-26T04:07:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:12 dist-test-slave-j8c1 krb5kdc[22016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:12 dist-test-slave-j8c1 krb5kdc[22016](info): set up 2 sockets
Sep 26 04:07:12 dist-test-slave-j8c1 krb5kdc[22016](info): commencing operation
krb5kdc: starting...
W20260926 04:07:13.346800 802 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:13.469458 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35957
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:13.551235 22029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:13.551465 22029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:13.551489 22029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:13.552698 22029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:13.552748 22029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:13.552763 22029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:13.552776 22029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:13.552788 22029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:13.554318 22029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:13.554620 22029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:13.554860 22029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:13.559844 22035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:13.559832 22034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:13.559828 22037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:13.560124 22029 server_base.cc:1034] running on GCE node
I20260926 04:07:13.560478 22029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:13.560791 22029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:13.561933 22029 hybrid_clock.cc:648] HybridClock initialized: now 1790395633561919 us; error 32 us; skew 500 ppm
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395633, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:13.570345 22029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:13.570990 22029 webserver.cc:460] Webserver started at http://127.0.0.1:38731/ using document root <none> and password file <none>
I20260926 04:07:13.571236 22029 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:13.571305 22029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:13.571449 22029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:13.572573 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "2445c6866f2c49c5918ac7659fbf9b62"
format_stamp: "Formatted at 2026-09-26 04:07:13 on dist-test-slave-j8c1"
server_key: "b72b69f6c5bb90dca21378064a2f81bf"
server_key_iv: "27fe692df17afe020a8c36a1327f0580"
server_key_version: "encryptionkey@0"
I20260926 04:07:13.572968 22029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "2445c6866f2c49c5918ac7659fbf9b62"
format_stamp: "Formatted at 2026-09-26 04:07:13 on dist-test-slave-j8c1"
server_key: "b72b69f6c5bb90dca21378064a2f81bf"
server_key_iv: "27fe692df17afe020a8c36a1327f0580"
server_key_version: "encryptionkey@0"
I20260926 04:07:13.574445 22029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:07:13.575390 22044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:13.575693 22029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:13.575831 22029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "2445c6866f2c49c5918ac7659fbf9b62"
format_stamp: "Formatted at 2026-09-26 04:07:13 on dist-test-slave-j8c1"
server_key: "b72b69f6c5bb90dca21378064a2f81bf"
server_key_iv: "27fe692df17afe020a8c36a1327f0580"
server_key_version: "encryptionkey@0"
I20260926 04:07:13.575922 22029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:13.592015 22029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:13.594905 22029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:13.595112 22029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:13.599246 22029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35957
I20260926 04:07:13.599283 22096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35957 every 8 connection(s)
I20260926 04:07:13.599651 22029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:13.600536 22097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:13.602942 22097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62: Bootstrap starting.
I20260926 04:07:13.603492 22097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:13.603739 22097 log.cc:824] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:13.604422 22097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62: No bootstrap required, opened a new log
I20260926 04:07:13.604722 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22029
I20260926 04:07:13.604843 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:13.605048 802 external_mini_cluster.cc:1428] Setting key 9d0143dcef91baf68839522c6005ab95
I20260926 04:07:13.606962 22097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } }
I20260926 04:07:13.607164 22097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:13.607219 22097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2445c6866f2c49c5918ac7659fbf9b62, State: Initialized, Role: FOLLOWER
I20260926 04:07:13.607357 22097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } }
I20260926 04:07:13.607486 22097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:13.607513 22097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:13.607568 22097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:13.608163 22097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } }
I20260926 04:07:13.608251 22097 leader_election.cc:302] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2445c6866f2c49c5918ac7659fbf9b62; no voters:
I20260926 04:07:13.608469 22097 leader_election.cc:288] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:13.608498 22102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:13.608611 22102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [term 1 LEADER]: Becoming Leader. State: Replica: 2445c6866f2c49c5918ac7659fbf9b62, State: Running, Role: LEADER
I20260926 04:07:13.608758 22097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:13.608758 22102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } }
I20260926 04:07:13.609326 22103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2445c6866f2c49c5918ac7659fbf9b62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } } }
I20260926 04:07:13.609467 22103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:13.609935 22111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:13.610395 22104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2445c6866f2c49c5918ac7659fbf9b62. Latest consensus state: current_term: 1 leader_uuid: "2445c6866f2c49c5918ac7659fbf9b62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2445c6866f2c49c5918ac7659fbf9b62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35957 } } }
I20260926 04:07:13.610461 22111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:13.611670 22104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62 [sys.catalog]: This master's current role is: LEADER
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 04:07:13.613612 22111 catalog_manager.cc:1347] Generated new cluster ID: fee478f8604a491182799b444194e2c9
I20260926 04:07:13.613668 22111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:13.617573 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:13.605590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:52535 (local address 127.0.0.1:35957)
0926 04:07:13.605776 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:13.605784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:13.605809 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:07:13.606391 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:13.606400 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:13.606453 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:13.606638 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:13.607446 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:13.609099 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:13.610299 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:13.610914 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:13.611029 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:13.613660 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:13.613664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:13.613664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:13.613677 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:13.615671 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:13.616117 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:13.616120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:13.616121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:13.616225 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:13.616430 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:13.616430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:13.616430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:13.616723 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:13.616860 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:13.617271 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 04:07:13.617370 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260926 04:07:13.618615 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:13.627525 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:13.620041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:33145 (local address 127.0.0.1:35957)
0926 04:07:13.620084 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:13.620087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:13.620160 (+ 73us) server_negotiation.cc:407] Connection header received
0926 04:07:13.620247 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:13.620247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:13.620258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:13.620284 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:13.620948 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:13.621706 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:13.622693 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:13.623041 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:13.623120 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:13.624526 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:13.624527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:13.624528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:13.624535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:13.626157 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:13.626634 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:13.626635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:13.626635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:13.626698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:13.626938 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:13.626939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:13.626939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:13.627082 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:13.627194 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:13.627341 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 04:07:13.627421 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:13.629740 22111 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:13.630278 22111 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:13.634855 22111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2445c6866f2c49c5918ac7659fbf9b62: Generated new TSK 0
I20260926 04:07:13.635121 22111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:13.640977 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22029
W20260926 04:07:13.645480 22014 connection.cc:535] client connection to 127.0.0.1:35957 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35957): Connection reset by peer (error 104)
2026-09-26T04:07:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22128](info): set up 2 sockets
Sep 26 04:07:13 dist-test-slave-j8c1 krb5kdc[22128](info): commencing operation
krb5kdc: starting...
W20260926 04:07:14.686867 802 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:14 dist-test-slave-j8c1 krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:14.809638 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:14.892463 22141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:14.892714 22141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:14.892740 22141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:14.893990 22141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:14.894044 22141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:14.894058 22141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:14.894073 22141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:14.894136 22141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:14.895815 22141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:14.896124 22141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:14.896380 22141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:14.901418 22146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:14.901464 22147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:14.901418 22149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:14.901631 22141 server_base.cc:1034] running on GCE node
I20260926 04:07:14.901877 22141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:14.902182 22141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:14.903378 22141 hybrid_clock.cc:648] HybridClock initialized: now 1790395634903352 us; error 36 us; skew 500 ppm
Sep 26 04:07:14 dist-test-slave-j8c1 krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:14.905987 22141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:14.906443 22141 webserver.cc:460] Webserver started at http://127.0.0.1:41321/ using document root <none> and password file <none>
I20260926 04:07:14.906752 22141 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:14.906824 22141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:14.906975 22141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:14.908056 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "9686481497584a5397cf48f8a480b6c0"
format_stamp: "Formatted at 2026-09-26 04:07:14 on dist-test-slave-j8c1"
server_key: "bafcdd3ad6ebe0b1d57b1e3a7d79c879"
server_key_iv: "88c0f9736b87856ee8d8fa051c6850ae"
server_key_version: "encryptionkey@0"
I20260926 04:07:14.908439 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "9686481497584a5397cf48f8a480b6c0"
format_stamp: "Formatted at 2026-09-26 04:07:14 on dist-test-slave-j8c1"
server_key: "bafcdd3ad6ebe0b1d57b1e3a7d79c879"
server_key_iv: "88c0f9736b87856ee8d8fa051c6850ae"
server_key_version: "encryptionkey@0"
I20260926 04:07:14.909888 22141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:14.910822 22156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:14.911026 22141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:14.911157 22141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "9686481497584a5397cf48f8a480b6c0"
format_stamp: "Formatted at 2026-09-26 04:07:14 on dist-test-slave-j8c1"
server_key: "bafcdd3ad6ebe0b1d57b1e3a7d79c879"
server_key_iv: "88c0f9736b87856ee8d8fa051c6850ae"
server_key_version: "encryptionkey@0"
I20260926 04:07:14.911242 22141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:14.941612 22141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:14.943898 22141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:14.944078 22141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:14.948338 22141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38803
I20260926 04:07:14.948377 22208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38803 every 8 connection(s)
I20260926 04:07:14.948750 22141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:14.949803 22209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:14.952145 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0: Bootstrap starting.
I20260926 04:07:14.952708 22209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:14.952965 22209 log.cc:824] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:14.953670 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0: No bootstrap required, opened a new log
I20260926 04:07:14.955216 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22141
I20260926 04:07:14.955360 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:14.955307 22209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } }
I20260926 04:07:14.955531 22209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:14.955592 22209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9686481497584a5397cf48f8a480b6c0, State: Initialized, Role: FOLLOWER
I20260926 04:07:14.955624 802 external_mini_cluster.cc:1428] Setting key 90d6f710fcc1ca9bff5134105753e253
I20260926 04:07:14.955750 22209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } }
I20260926 04:07:14.955850 22209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:14.955888 22209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:14.955940 22209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:14.957130 22209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } }
I20260926 04:07:14.957260 22209 leader_election.cc:302] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9686481497584a5397cf48f8a480b6c0; no voters:
I20260926 04:07:14.957448 22209 leader_election.cc:288] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:14.957476 22214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:14.957610 22214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [term 1 LEADER]: Becoming Leader. State: Replica: 9686481497584a5397cf48f8a480b6c0, State: Running, Role: LEADER
I20260926 04:07:14.957731 22209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:14.957746 22214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } }
I20260926 04:07:14.958145 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9686481497584a5397cf48f8a480b6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } } }
I20260926 04:07:14.958230 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:14.958461 22220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:14.959020 22220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:14.959306 22216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9686481497584a5397cf48f8a480b6c0. Latest consensus state: current_term: 1 leader_uuid: "9686481497584a5397cf48f8a480b6c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9686481497584a5397cf48f8a480b6c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38803 } } }
I20260926 04:07:14.959677 22216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:14.960970 22220 catalog_manager.cc:1347] Generated new cluster ID: 19973985e7204d62931efb47aed99ff0
I20260926 04:07:14.961023 22220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:14.961938 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:14.956201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:58449 (local address 127.0.0.1:38803)
0926 04:07:14.956376 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:14.956386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:14.956422 (+ 36us) server_negotiation.cc:407] Connection header received
0926 04:07:14.956548 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:14.956553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:14.956617 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:14.956730 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:14.956981 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:14.958275 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:14.959570 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:14.960162 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:14.960598 (+ 436us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:14.961087 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:14.961091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:14.961091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:14.961302 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:14.961340 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:14.961356 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:14.961657 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:14.961702 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 04:07:14.961729 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260926 04:07:14.963080 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:14.968151 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:14.964628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:38177 (local address 127.0.0.1:38803)
0926 04:07:14.964662 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:14.964664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:14.964694 (+ 30us) server_negotiation.cc:407] Connection header received
0926 04:07:14.964716 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:14.964717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:14.964727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:14.964779 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:14.964926 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:14.965706 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:14.966725 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:14.967047 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:14.967131 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:14.967878 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:14.967879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:14.967879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:14.967884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:14.967901 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:14.967906 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:14.967968 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:14.968047 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 04:07:14.968071 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 04:07:14.977859 22220 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:14.978439 22220 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:14.982470 22220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9686481497584a5397cf48f8a480b6c0: Generated new TSK 0
I20260926 04:07:14.982712 22220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:14.988523 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22141
W20260926 04:07:14.993304 22126 connection.cc:535] client connection to 127.0.0.1:38803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38803): Connection reset by peer (error 104)
2026-09-26T04:07:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:15 dist-test-slave-j8c1 krb5kdc[22240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:15 dist-test-slave-j8c1 krb5kdc[22240](info): set up 2 sockets
Sep 26 04:07:15 dist-test-slave-j8c1 krb5kdc[22240](info): commencing operation
krb5kdc: starting...
W20260926 04:07:16.070672 802 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:16 dist-test-slave-j8c1 krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 04:07:16.190634 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:16.270442 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:16.270746 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:16.270788 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:16.272015 22253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:16.272111 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:16.272137 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:16.272166 22253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:16.272195 22253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:16.273731 22253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:16.274026 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:16.274262 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:16.279429 22258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:16.279497 22261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:16.279533 22253 server_base.cc:1034] running on GCE node
W20260926 04:07:16.279429 22259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:16.279912 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:16.280224 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:16.281457 22253 hybrid_clock.cc:648] HybridClock initialized: now 1790395636281445 us; error 32 us; skew 500 ppm
Sep 26 04:07:16 dist-test-slave-j8c1 krb5kdc[22240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:16.284129 22253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 04:07:16.284660 22253 webserver.cc:460] Webserver started at http://127.0.0.1:32797/ using document root <none> and password file <none>
I20260926 04:07:16.284895 22253 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:16.284958 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:16.285084 22253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:16.286141 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "5b4435590af3489995a830547e3f7683"
format_stamp: "Formatted at 2026-09-26 04:07:16 on dist-test-slave-j8c1"
server_key: "aedc68592b2cebeb3f45d306a7b62779"
server_key_iv: "ecbda2c60a78384468e54e9d6873664b"
server_key_version: "encryptionkey@0"
I20260926 04:07:16.286453 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "5b4435590af3489995a830547e3f7683"
format_stamp: "Formatted at 2026-09-26 04:07:16 on dist-test-slave-j8c1"
server_key: "aedc68592b2cebeb3f45d306a7b62779"
server_key_iv: "ecbda2c60a78384468e54e9d6873664b"
server_key_version: "encryptionkey@0"
I20260926 04:07:16.287832 22253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:16.288578 22268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:16.288765 22253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:16.288870 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "5b4435590af3489995a830547e3f7683"
format_stamp: "Formatted at 2026-09-26 04:07:16 on dist-test-slave-j8c1"
server_key: "aedc68592b2cebeb3f45d306a7b62779"
server_key_iv: "ecbda2c60a78384468e54e9d6873664b"
server_key_version: "encryptionkey@0"
I20260926 04:07:16.288946 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:16.292510 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:16.294691 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:16.294847 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:16.298964 22253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32863
I20260926 04:07:16.298997 22320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32863 every 8 connection(s)
I20260926 04:07:16.299366 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:16.300390 22321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:16.302778 22321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683: Bootstrap starting.
I20260926 04:07:16.303308 22321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:16.303557 22321 log.cc:824] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:16.304226 22321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683: No bootstrap required, opened a new log
I20260926 04:07:16.305383 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22253
I20260926 04:07:16.305528 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:16.305729 802 external_mini_cluster.cc:1428] Setting key 84f642730106c1c1156ff92c8d9c0d53
I20260926 04:07:16.305939 22321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } }
I20260926 04:07:16.306098 22321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:16.306133 22321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4435590af3489995a830547e3f7683, State: Initialized, Role: FOLLOWER
I20260926 04:07:16.306249 22321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } }
I20260926 04:07:16.306361 22321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:16.306433 22321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:16.306521 22321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:16.307813 22321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } }
I20260926 04:07:16.307929 22321 leader_election.cc:302] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b4435590af3489995a830547e3f7683; no voters:
I20260926 04:07:16.308086 22325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:16.306320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41675 (local address 127.0.0.1:32863)
0926 04:07:16.306520 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:16.306529 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:16.306583 (+ 54us) server_negotiation.cc:407] Connection header received
0926 04:07:16.306733 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:16.306739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:16.306763 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:16.306882 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:16.307566 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:16.307569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:16.307569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:16.307573 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:16.307611 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:16.307622 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:16.307677 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:16.307772 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 04:07:16.307835 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260926 04:07:16.308126 22326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:16.308094 22321 leader_election.cc:288] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:16.308286 22326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [term 1 LEADER]: Becoming Leader. State: Replica: 5b4435590af3489995a830547e3f7683, State: Running, Role: LEADER
I20260926 04:07:16.308501 22326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } }
I20260926 04:07:16.308976 22326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b4435590af3489995a830547e3f7683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } } }
I20260926 04:07:16.309059 22326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:16.309088 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:16.309087 22327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b4435590af3489995a830547e3f7683. Latest consensus state: current_term: 1 leader_uuid: "5b4435590af3489995a830547e3f7683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4435590af3489995a830547e3f7683" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32863 } } }
I20260926 04:07:16.309195 22327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:16.308549 22321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:16.309310 22330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:16.309845 22330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:16.311983 22325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:16.310728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:56313 (local address 127.0.0.1:32863)
0926 04:07:16.310832 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:16.310834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:16.310975 (+ 141us) server_negotiation.cc:407] Connection header received
0926 04:07:16.311020 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:16.311021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:16.311027 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:16.311087 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:16.311774 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:16.311776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:16.311776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:16.311780 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:16.311795 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:16.311800 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:16.311871 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:16.311912 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 04:07:16.311927 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 04:07:16.315663 22330 catalog_manager.cc:1347] Generated new cluster ID: 4a4374a0e630446ab4de9898d0a2d1fc
I20260926 04:07:16.315729 22330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:16.324524 22330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:16.324731 22330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:16.327225 22330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b4435590af3489995a830547e3f7683: Generated new TSK 0
I20260926 04:07:16.327437 22330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:16.337548 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22253
2026-09-26T04:07:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:16 dist-test-slave-j8c1 krb5kdc[22352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:16 dist-test-slave-j8c1 krb5kdc[22352](info): set up 2 sockets
Sep 26 04:07:16 dist-test-slave-j8c1 krb5kdc[22352](info): commencing operation
krb5kdc: starting...
W20260926 04:07:17.374876 802 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:17.502499 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43929
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:17.584189 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:17.584482 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:17.584529 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:17.586220 22365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:17.586336 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:17.586365 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:17.586388 22365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:17.586419 22365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:17.588109 22365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43929
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:17.588428 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:17.588680 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:17.594130 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:17.594259 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:17.594200 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:17.594457 22365 server_base.cc:1034] running on GCE node
I20260926 04:07:17.594784 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:17.595112 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:17.596320 22365 hybrid_clock.cc:648] HybridClock initialized: now 1790395637596282 us; error 65 us; skew 500 ppm
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395637, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:17.599427 22365 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:17.599884 22365 webserver.cc:460] Webserver started at http://127.0.200.190:39967/ using document root <none> and password file <none>
I20260926 04:07:17.600098 22365 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:17.600142 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:17.600236 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:17.601306 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "9ba3805799a9402b822e2a4cc299732a"
format_stamp: "Formatted at 2026-09-26 04:07:17 on dist-test-slave-j8c1"
server_key: "a88a8cdecf43f9139771434ff2b2e235"
server_key_iv: "58747a785735134f032aa604d07f34cd"
server_key_version: "encryptionkey@0"
I20260926 04:07:17.601637 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "9ba3805799a9402b822e2a4cc299732a"
format_stamp: "Formatted at 2026-09-26 04:07:17 on dist-test-slave-j8c1"
server_key: "a88a8cdecf43f9139771434ff2b2e235"
server_key_iv: "58747a785735134f032aa604d07f34cd"
server_key_version: "encryptionkey@0"
I20260926 04:07:17.603190 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:17.604071 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:17.604355 22365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:17.604511 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "9ba3805799a9402b822e2a4cc299732a"
format_stamp: "Formatted at 2026-09-26 04:07:17 on dist-test-slave-j8c1"
server_key: "a88a8cdecf43f9139771434ff2b2e235"
server_key_iv: "58747a785735134f032aa604d07f34cd"
server_key_version: "encryptionkey@0"
I20260926 04:07:17.604607 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:17.633234 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:17.635756 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:17.635932 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:17.640122 22365 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43929
I20260926 04:07:17.640170 22432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43929 every 8 connection(s)
I20260926 04:07:17.640543 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:17.641400 22433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:17.643671 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a: Bootstrap starting.
I20260926 04:07:17.644268 22433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:17.644553 22433 log.cc:824] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:17.645304 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a: No bootstrap required, opened a new log
I20260926 04:07:17.646952 22433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } }
I20260926 04:07:17.647123 22433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:17.647147 22433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba3805799a9402b822e2a4cc299732a, State: Initialized, Role: FOLLOWER
I20260926 04:07:17.647310 22433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } }
I20260926 04:07:17.647372 22433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:17.647393 22433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:17.647461 22433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:17.648006 22433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } }
I20260926 04:07:17.648089 22433 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba3805799a9402b822e2a4cc299732a; no voters:
I20260926 04:07:17.648299 22433 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:17.648345 22436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:17.648414 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22365
I20260926 04:07:17.648536 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:17.648572 22433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:17.648577 22436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [term 1 LEADER]: Becoming Leader. State: Replica: 9ba3805799a9402b822e2a4cc299732a, State: Running, Role: LEADER
I20260926 04:07:17.648701 22436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } }
I20260926 04:07:17.648767 802 external_mini_cluster.cc:1428] Setting key 82a0a6f4e569d339bd5b6965d898c81f
I20260926 04:07:17.649204 22437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba3805799a9402b822e2a4cc299732a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } } }
I20260926 04:07:17.649372 22437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:17.649757 22438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba3805799a9402b822e2a4cc299732a. Latest consensus state: current_term: 1 leader_uuid: "9ba3805799a9402b822e2a4cc299732a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba3805799a9402b822e2a4cc299732a" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43929 } } }
I20260926 04:07:17.649928 22438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:17.649932 22452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:17.650821 22452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:17.652383 22452 catalog_manager.cc:1347] Generated new cluster ID: 889af282d66d416997963680068b7286
I20260926 04:07:17.652438 22452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:17.660117 22452 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:17.660755 22452 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:17.661433 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:17.649419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49375 (local address 127.0.200.190:43929)
0926 04:07:17.649987 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:17.649997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:17.650031 (+ 34us) server_negotiation.cc:407] Connection header received
0926 04:07:17.650153 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:17.650161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:17.650223 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:17.650381 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:17.651305 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:17.652696 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:17.653763 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:17.654406 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:17.654574 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:17.657432 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:17.657439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:17.657439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:17.657467 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:17.659670 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:17.660159 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:17.660163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:17.660163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:17.660270 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:17.660534 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:17.660535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:17.660536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:17.660830 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:17.661001 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:17.661112 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 04:07:17.661218 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":122,"threads_started":1}
I20260926 04:07:17.662448 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:17.669020 22452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba3805799a9402b822e2a4cc299732a: Generated new TSK 0
I20260926 04:07:17.669195 22452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:17.673715 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:17.663815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51391 (local address 127.0.200.190:43929)
0926 04:07:17.663853 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:17.663855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:17.666740 (+ 2885us) server_negotiation.cc:407] Connection header received
0926 04:07:17.666772 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:17.666773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:17.666784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:17.666850 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:17.667468 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:17.668286 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:17.669213 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:17.669561 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:17.669632 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:17.670892 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:17.670893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:17.670893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:17.670900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:17.672417 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:17.672814 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:17.672815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:17.672816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:17.672876 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:17.673152 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:17.673152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:17.673153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:17.673281 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:17.673388 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:17.673550 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 04:07:17.673617 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:07:17.676064 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22365
W20260926 04:07:17.680308 22350 connection.cc:535] client connection to 127.0.200.190:43929 recv error: Network error: failed to read from TLS socket (remote: 127.0.200.190:43929): Connection reset by peer (error 104)
2026-09-26T04:07:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22464](info): set up 2 sockets
Sep 26 04:07:17 dist-test-slave-j8c1 krb5kdc[22464](info): commencing operation
krb5kdc: starting...
W20260926 04:07:18.738727 802 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:18 dist-test-slave-j8c1 krb5kdc[22464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:18.858134 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43379
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:18.937593 22477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:18.937863 22477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:18.937904 22477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:18.939086 22477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:18.939155 22477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:18.939180 22477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:18.939208 22477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:18.939239 22477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:18.940680 22477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:18.940955 22477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:18.941184 22477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:18.946101 22483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:18.946113 22485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:18.946324 22477 server_base.cc:1034] running on GCE node
W20260926 04:07:18.946137 22482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:18.946676 22477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:18.947000 22477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:18.948190 22477 hybrid_clock.cc:648] HybridClock initialized: now 1790395638948174 us; error 27 us; skew 500 ppm
Sep 26 04:07:18 dist-test-slave-j8c1 krb5kdc[22464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:18.957171 22477 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:18.957628 22477 webserver.cc:460] Webserver started at http://127.0.200.190:37497/ using document root <none> and password file <none>
I20260926 04:07:18.957898 22477 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:18.957970 22477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:18.958119 22477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:18.959331 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e"
format_stamp: "Formatted at 2026-09-26 04:07:18 on dist-test-slave-j8c1"
server_key: "2905d8deaa0d35450711e3d0f65cf147"
server_key_iv: "999184a00ed268bbe319d051b96fa024"
server_key_version: "encryptionkey@0"
I20260926 04:07:18.959713 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e"
format_stamp: "Formatted at 2026-09-26 04:07:18 on dist-test-slave-j8c1"
server_key: "2905d8deaa0d35450711e3d0f65cf147"
server_key_iv: "999184a00ed268bbe319d051b96fa024"
server_key_version: "encryptionkey@0"
I20260926 04:07:18.961097 22477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:07:18.961869 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:18.962044 22477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260926 04:07:18.962144 22477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e"
format_stamp: "Formatted at 2026-09-26 04:07:18 on dist-test-slave-j8c1"
server_key: "2905d8deaa0d35450711e3d0f65cf147"
server_key_iv: "999184a00ed268bbe319d051b96fa024"
server_key_version: "encryptionkey@0"
I20260926 04:07:18.962246 22477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:18.976099 22477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:18.978227 22477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:18.978422 22477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:18.982275 22477 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43379
I20260926 04:07:18.982347 22544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43379 every 8 connection(s)
I20260926 04:07:18.982776 22477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:18.983628 22545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:18.985934 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e: Bootstrap starting.
I20260926 04:07:18.986482 22545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:18.986820 22545 log.cc:824] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:18.987494 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e: No bootstrap required, opened a new log
I20260926 04:07:18.989030 22545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } }
I20260926 04:07:18.989168 22545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:18.989192 22545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0750e4a0ba4cf4bfa4e80a65a76c9e, State: Initialized, Role: FOLLOWER
I20260926 04:07:18.989339 22545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } }
I20260926 04:07:18.989423 22545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:18.989446 22545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:18.989476 22545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:18.989955 22545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } }
I20260926 04:07:18.990041 22545 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0750e4a0ba4cf4bfa4e80a65a76c9e; no voters:
I20260926 04:07:18.990159 22545 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:18.990248 22548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:18.990413 22548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [term 1 LEADER]: Becoming Leader. State: Replica: 1b0750e4a0ba4cf4bfa4e80a65a76c9e, State: Running, Role: LEADER
I20260926 04:07:18.990451 22545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:18.990514 22548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } }
I20260926 04:07:18.991112 22550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0750e4a0ba4cf4bfa4e80a65a76c9e. Latest consensus state: current_term: 1 leader_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } } }
I20260926 04:07:18.991202 22550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:18.991392 22549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0750e4a0ba4cf4bfa4e80a65a76c9e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43379 } } }
I20260926 04:07:18.991520 22549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:18.991573 22557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:18.992102 22557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:18.992895 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22477
I20260926 04:07:18.993031 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:18.993233 802 external_mini_cluster.cc:1428] Setting key 032ff2f480271f6f2d3bc9fadc76db6d
I20260926 04:07:18.993696 22557 catalog_manager.cc:1347] Generated new cluster ID: b3df498693754b5bbacc3bb44645f488
I20260926 04:07:18.993747 22557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:18.999511 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:18.994007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45863 (local address 127.0.200.190:43379)
0926 04:07:18.994168 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:18.994177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:18.994206 (+ 29us) server_negotiation.cc:407] Connection header received
0926 04:07:18.994314 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:18.994320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:18.994370 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:18.994491 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:18.994768 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:18.996140 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:18.997357 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:18.997931 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:18.998027 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:18.998849 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:18.998853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:18.998853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:18.999051 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:18.999087 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:18.999101 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:18.999178 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:18.999290 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 04:07:18.999314 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260926 04:07:19.000414 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:19.001631 22557 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:19.002635 22557 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:19.005421 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:19.001916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45623 (local address 127.0.200.190:43379)
0926 04:07:19.001952 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:19.001954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:19.002016 (+ 62us) server_negotiation.cc:407] Connection header received
0926 04:07:19.002105 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:19.002106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:19.002116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:19.002184 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:19.002309 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:19.003009 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:19.004028 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:19.004324 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:19.004383 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:19.005077 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:19.005078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:19.005078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:19.005082 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:19.005096 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:19.005100 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:19.005154 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:19.005253 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 04:07:19.005339 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:19.013181 22557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0750e4a0ba4cf4bfa4e80a65a76c9e: Generated new TSK 0
I20260926 04:07:19.013375 22557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:19.017462 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22477
W20260926 04:07:19.021733 22462 connection.cc:535] client connection to 127.0.200.190:43379 recv error: Network error: failed to read from TLS socket (remote: 127.0.200.190:43379): Connection reset by peer (error 104)
2026-09-26T04:07:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:19 dist-test-slave-j8c1 krb5kdc[22576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:19 dist-test-slave-j8c1 krb5kdc[22576](info): set up 2 sockets
Sep 26 04:07:19 dist-test-slave-j8c1 krb5kdc[22576](info): commencing operation
krb5kdc: starting...
W20260926 04:07:20.078732 802 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:20 dist-test-slave-j8c1 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:20.217703 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40125
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:40125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:20.298000 22589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:20.298235 22589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:20.298260 22589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:20.299814 22589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:20.299886 22589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:20.299902 22589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:20.299916 22589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:20.299928 22589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:20.301574 22589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:40125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:40125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:20.301803 22589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:20.302093 22589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:20.306944 22594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:20.306972 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:20.307040 22595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:20.307703 22589 server_base.cc:1034] running on GCE node
I20260926 04:07:20.308005 22589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:20.308334 22589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:20.309521 22589 hybrid_clock.cc:648] HybridClock initialized: now 1790395640309511 us; error 54 us; skew 500 ppm
Sep 26 04:07:20 dist-test-slave-j8c1 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:20.312201 22589 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:20.312662 22589 webserver.cc:460] Webserver started at http://127.0.200.190:41123/ using document root <none> and password file <none>
I20260926 04:07:20.312912 22589 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:20.312984 22589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:20.313138 22589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:20.314222 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "82394926b7e743efb877ad9f880e0780"
format_stamp: "Formatted at 2026-09-26 04:07:20 on dist-test-slave-j8c1"
server_key: "ddd61bce766a4975ec7502df0f05191c"
server_key_iv: "7772abf2c286e30a7431e516d041b9ab"
server_key_version: "encryptionkey@0"
I20260926 04:07:20.314684 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "82394926b7e743efb877ad9f880e0780"
format_stamp: "Formatted at 2026-09-26 04:07:20 on dist-test-slave-j8c1"
server_key: "ddd61bce766a4975ec7502df0f05191c"
server_key_iv: "7772abf2c286e30a7431e516d041b9ab"
server_key_version: "encryptionkey@0"
I20260926 04:07:20.316221 22589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:20.317039 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:20.317279 22589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:20.317395 22589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "82394926b7e743efb877ad9f880e0780"
format_stamp: "Formatted at 2026-09-26 04:07:20 on dist-test-slave-j8c1"
server_key: "ddd61bce766a4975ec7502df0f05191c"
server_key_iv: "7772abf2c286e30a7431e516d041b9ab"
server_key_version: "encryptionkey@0"
I20260926 04:07:20.317476 22589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:20.320935 22589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:20.323324 22589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:20.323490 22589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:20.327327 22589 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:40125
I20260926 04:07:20.327358 22656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:40125 every 8 connection(s)
I20260926 04:07:20.327690 22589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:20.328699 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:20.331123 22657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780: Bootstrap starting.
I20260926 04:07:20.331700 22657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:20.331984 22657 log.cc:824] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:20.332455 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22589
I20260926 04:07:20.332576 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:20.332768 802 external_mini_cluster.cc:1428] Setting key f7fc31e45c40635fc65f28f5252f3336
I20260926 04:07:20.332829 22657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780: No bootstrap required, opened a new log
I20260926 04:07:20.334743 22657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } }
I20260926 04:07:20.334908 22657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:20.334950 22657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82394926b7e743efb877ad9f880e0780, State: Initialized, Role: FOLLOWER
I20260926 04:07:20.335140 22657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } }
I20260926 04:07:20.335237 22657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:20.335286 22657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:20.335335 22657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:20.336170 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:20.333284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54355 (local address 127.0.200.190:40125)
0926 04:07:20.333473 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:20.333482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:20.333506 (+ 24us) server_negotiation.cc:407] Connection header received
0926 04:07:20.334733 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:20.334741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:20.334772 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:20.334895 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:20.335665 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:20.335669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:20.335669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:20.335673 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:20.335713 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:20.335724 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:20.335787 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:20.335877 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 04:07:20.335910 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260926 04:07:20.336537 22657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } }
I20260926 04:07:20.336690 22657 leader_election.cc:302] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82394926b7e743efb877ad9f880e0780; no voters:
I20260926 04:07:20.336900 22657 leader_election.cc:288] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:20.336942 22662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:20.337096 22662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [term 1 LEADER]: Becoming Leader. State: Replica: 82394926b7e743efb877ad9f880e0780, State: Running, Role: LEADER
I20260926 04:07:20.337198 22657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:20.337189 22662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } }
I20260926 04:07:20.337548 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82394926b7e743efb877ad9f880e0780. Latest consensus state: current_term: 1 leader_uuid: "82394926b7e743efb877ad9f880e0780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } } }
I20260926 04:07:20.337551 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82394926b7e743efb877ad9f880e0780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82394926b7e743efb877ad9f880e0780" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 40125 } } }
I20260926 04:07:20.337637 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:20.337725 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:20.338029 22668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:20.338153 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:20.338627 22668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:20.340935 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:20.339738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47577 (local address 127.0.200.190:40125)
0926 04:07:20.339780 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:20.339782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:20.339910 (+ 128us) server_negotiation.cc:407] Connection header received
0926 04:07:20.339988 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:20.339989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:20.339994 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:20.340050 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:20.340718 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:20.340719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:20.340719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:20.340724 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:20.340739 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:20.340743 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:20.340800 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:20.340855 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 04:07:20.340870 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:20.341080 22668 catalog_manager.cc:1347] Generated new cluster ID: 28bd72c9602b4393a820d9048327523d
I20260926 04:07:20.341128 22668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:20.368114 22668 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:20.368305 22668 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:20.372941 22668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82394926b7e743efb877ad9f880e0780: Generated new TSK 0
I20260926 04:07:20.373129 22668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:20.377259 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22589
2026-09-26T04:07:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:20 dist-test-slave-j8c1 krb5kdc[22688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:20 dist-test-slave-j8c1 krb5kdc[22688](info): set up 2 sockets
Sep 26 04:07:20 dist-test-slave-j8c1 krb5kdc[22688](info): commencing operation
krb5kdc: starting...
W20260926 04:07:21.406670 802 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:21.525645 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37233
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:37233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:21.606794 22701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:21.607023 22701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:21.607060 22701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:21.608223 22701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:21.608289 22701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:21.608322 22701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:21.608354 22701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:21.608379 22701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:21.610211 22701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:37233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:21.610512 22701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:21.610757 22701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:21.615669 22707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:21.615665 22709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:21.615638 22706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:21.615916 22701 server_base.cc:1034] running on GCE node
I20260926 04:07:21.616168 22701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:21.616454 22701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:21.617623 22701 hybrid_clock.cc:648] HybridClock initialized: now 1790395641617617 us; error 36 us; skew 500 ppm
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:21.620139 22701 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:21.620611 22701 webserver.cc:460] Webserver started at http://127.0.200.190:37561/ using document root <none> and password file <none>
I20260926 04:07:21.620834 22701 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:21.620879 22701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:21.620967 22701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:21.621986 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "9c4022e531384519b03db897e755dbd8"
format_stamp: "Formatted at 2026-09-26 04:07:21 on dist-test-slave-j8c1"
server_key: "7265d5c4720fcb799cebb53659ee8849"
server_key_iv: "3cafa829fa4a0998fdc417da12cd8c8b"
server_key_version: "encryptionkey@0"
I20260926 04:07:21.622283 22701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "9c4022e531384519b03db897e755dbd8"
format_stamp: "Formatted at 2026-09-26 04:07:21 on dist-test-slave-j8c1"
server_key: "7265d5c4720fcb799cebb53659ee8849"
server_key_iv: "3cafa829fa4a0998fdc417da12cd8c8b"
server_key_version: "encryptionkey@0"
I20260926 04:07:21.623641 22701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:21.624397 22716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:21.624598 22701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 04:07:21.624691 22701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "9c4022e531384519b03db897e755dbd8"
format_stamp: "Formatted at 2026-09-26 04:07:21 on dist-test-slave-j8c1"
server_key: "7265d5c4720fcb799cebb53659ee8849"
server_key_iv: "3cafa829fa4a0998fdc417da12cd8c8b"
server_key_version: "encryptionkey@0"
I20260926 04:07:21.624768 22701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:21.642021 22701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:21.643977 22701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:21.644150 22701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:21.648442 22701 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:37233
I20260926 04:07:21.648497 22768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:37233 every 8 connection(s)
I20260926 04:07:21.648847 22701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:21.649693 22769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:21.650332 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22701
I20260926 04:07:21.650424 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:21.650657 802 external_mini_cluster.cc:1428] Setting key 584fffee5825e153b6c19f1c73c4a263
I20260926 04:07:21.652796 22769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: Bootstrap starting.
I20260926 04:07:21.653365 22769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:21.653651 22769 log.cc:824] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:21.654331 22769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: No bootstrap required, opened a new log
I20260926 04:07:21.656376 22769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } }
I20260926 04:07:21.656533 22769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:21.656597 22769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4022e531384519b03db897e755dbd8, State: Initialized, Role: FOLLOWER
I20260926 04:07:21.656770 22769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } }
I20260926 04:07:21.656867 22769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:21.656910 22769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:21.656958 22769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:21.657537 22769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } }
I20260926 04:07:21.657653 22769 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c4022e531384519b03db897e755dbd8; no voters:
I20260926 04:07:21.657871 22769 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:21.657899 22774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:21.658079 22774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9c4022e531384519b03db897e755dbd8, State: Running, Role: LEADER
I20260926 04:07:21.658133 22769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:21.659757 22774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } }
I20260926 04:07:21.660331 22775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4022e531384519b03db897e755dbd8. Latest consensus state: current_term: 1 leader_uuid: "9c4022e531384519b03db897e755dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } } }
I20260926 04:07:21.660425 22775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:21.660516 22776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c4022e531384519b03db897e755dbd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4022e531384519b03db897e755dbd8" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 37233 } } }
I20260926 04:07:21.661029 22776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8 [sys.catalog]: This master's current role is: LEADER
W20260926 04:07:21.660758 22788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:21.661348 22788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:21.661774 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:21.651160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54107 (local address 127.0.200.190:37233)
0926 04:07:21.651328 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:21.651338 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:21.651370 (+ 32us) server_negotiation.cc:407] Connection header received
0926 04:07:21.651753 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:21.651758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:21.651800 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:21.651932 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:21.652735 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:21.653684 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:21.654643 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:21.655046 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:21.655175 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:21.657865 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:21.657869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:21.657869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:21.657880 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:21.660020 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:21.660434 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:21.660438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:21.660438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:21.660551 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:21.660849 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:21.660850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:21.660850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:21.661170 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:21.661347 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:21.661431 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 04:07:21.661565 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260926 04:07:21.662727 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:21.666694 22790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:21.666899 22790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:21.668417 22790 catalog_manager.cc:1347] Generated new cluster ID: 829b9d8560774cd79ee9c04d37b208ad
I20260926 04:07:21.668463 22790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:21.670771 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:21.664036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54591 (local address 127.0.200.190:37233)
0926 04:07:21.664078 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:21.664080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:21.664179 (+ 99us) server_negotiation.cc:407] Connection header received
0926 04:07:21.664262 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:21.664263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:21.664273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:21.664329 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:21.664831 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:21.665300 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:21.665974 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:21.666211 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:21.666285 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:21.668155 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:21.668156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:21.668156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:21.668163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:21.669692 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:21.670020 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:21.670020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:21.670021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:21.670065 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:21.670244 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:21.670244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:21.670244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:21.670354 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:21.670414 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:21.670624 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 04:07:21.670690 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:07:21.686529 22790 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:21.687048 22790 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:21.693998 22790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c4022e531384519b03db897e755dbd8: Generated new TSK 0
I20260926 04:07:21.694197 22790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:21.705350 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22701
2026-09-26T04:07:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22800](info): set up 2 sockets
Sep 26 04:07:21 dist-test-slave-j8c1 krb5kdc[22800](info): commencing operation
krb5kdc: starting...
W20260926 04:07:22.734946 802 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:22 dist-test-slave-j8c1 krb5kdc[22800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:22.870162 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36253
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:38465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:22.949632 22813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:22.949885 22813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:22.949922 22813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:22.951201 22813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:22.951264 22813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:22.951288 22813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:22.951315 22813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:22.951347 22813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:22.952782 22813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:38465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:22.953045 22813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:22.953265 22813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:22.957996 22819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:22.958252 22818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:22.958271 22821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:22.958284 22813 server_base.cc:1034] running on GCE node
I20260926 04:07:22.958666 22813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:22.958959 22813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:22.960125 22813 hybrid_clock.cc:648] HybridClock initialized: now 1790395642960113 us; error 44 us; skew 500 ppm
Sep 26 04:07:22 dist-test-slave-j8c1 krb5kdc[22800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:22.962621 22813 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:22.963073 22813 webserver.cc:460] Webserver started at http://127.0.200.190:35517/ using document root <none> and password file <none>
I20260926 04:07:22.963258 22813 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:22.963297 22813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:22.963385 22813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:22.964371 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "19c9b498dbd946148906539e0a93b20d"
format_stamp: "Formatted at 2026-09-26 04:07:22 on dist-test-slave-j8c1"
server_key: "2d69c4e8669683eafb4d6bb18bf1caf0"
server_key_iv: "19041e0bfd0b4ed778ef4d11ed1253ce"
server_key_version: "encryptionkey@0"
I20260926 04:07:22.964663 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "19c9b498dbd946148906539e0a93b20d"
format_stamp: "Formatted at 2026-09-26 04:07:22 on dist-test-slave-j8c1"
server_key: "2d69c4e8669683eafb4d6bb18bf1caf0"
server_key_iv: "19041e0bfd0b4ed778ef4d11ed1253ce"
server_key_version: "encryptionkey@0"
I20260926 04:07:22.965951 22813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:22.966688 22828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:22.966898 22813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:22.966970 22813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "19c9b498dbd946148906539e0a93b20d"
format_stamp: "Formatted at 2026-09-26 04:07:22 on dist-test-slave-j8c1"
server_key: "2d69c4e8669683eafb4d6bb18bf1caf0"
server_key_iv: "19041e0bfd0b4ed778ef4d11ed1253ce"
server_key_version: "encryptionkey@0"
I20260926 04:07:22.967022 22813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:22.992235 22813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:22.994275 22813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:22.994387 22813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:22.998260 22813 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:36253
I20260926 04:07:22.998322 22880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:36253 every 8 connection(s)
I20260926 04:07:22.998663 22813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:22.999595 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:23.001865 22881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d: Bootstrap starting.
I20260926 04:07:23.002439 22881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:23.002792 22881 log.cc:824] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:23.003494 22881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d: No bootstrap required, opened a new log
I20260926 04:07:23.005203 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22813
I20260926 04:07:23.005314 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:23.005260 22881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } }
I20260926 04:07:23.005437 22881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:23.005475 22881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19c9b498dbd946148906539e0a93b20d, State: Initialized, Role: FOLLOWER
I20260926 04:07:23.005510 802 external_mini_cluster.cc:1428] Setting key 0743eec24cbca9c0d167419ba1dbe0da
I20260926 04:07:23.005632 22881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } }
I20260926 04:07:23.005715 22881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:23.005762 22881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:23.005808 22881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:23.007176 22881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } }
I20260926 04:07:23.007289 22881 leader_election.cc:302] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19c9b498dbd946148906539e0a93b20d; no voters:
I20260926 04:07:23.007443 22881 leader_election.cc:288] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:23.007495 22886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:23.007643 22886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [term 1 LEADER]: Becoming Leader. State: Replica: 19c9b498dbd946148906539e0a93b20d, State: Running, Role: LEADER
I20260926 04:07:23.007679 22881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:23.007809 22886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } }
I20260926 04:07:23.008163 22887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19c9b498dbd946148906539e0a93b20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } } }
I20260926 04:07:23.008178 22888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19c9b498dbd946148906539e0a93b20d. Latest consensus state: current_term: 1 leader_uuid: "19c9b498dbd946148906539e0a93b20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19c9b498dbd946148906539e0a93b20d" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36253 } } }
I20260926 04:07:23.008364 22887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:23.008378 22888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:23.009076 22895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:23.010332 22895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:23.011228 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:23.005957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57649 (local address 127.0.200.190:36253)
0926 04:07:23.006134 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:23.006143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:23.006211 (+ 68us) server_negotiation.cc:407] Connection header received
0926 04:07:23.006328 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:23.006335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:23.006407 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:23.006582 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:23.006774 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:23.008192 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:23.009294 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:23.009687 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:23.009770 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:23.010492 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:23.010496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:23.010496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:23.010741 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:23.010778 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:23.010792 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:23.010864 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:23.010954 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 04:07:23.011018 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260926 04:07:23.011937 22895 catalog_manager.cc:1347] Generated new cluster ID: 10b9e601a74945ea92393ef3812113e1
I20260926 04:07:23.011986 22895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:23.012197 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:23.017850 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:23.013565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40503 (local address 127.0.200.190:36253)
0926 04:07:23.014673 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:23.014675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:23.014684 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:23.014711 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:23.014712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:23.014736 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:23.014798 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:23.014960 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:23.015673 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:23.016612 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:23.016904 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:23.016977 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:23.017605 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:23.017606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:23.017606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:23.017611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:23.017627 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:23.017631 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:23.017703 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:23.017747 (+ 44us) server_negotiation.cc:299] Negotiation successful
0926 04:07:23.017770 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089}
I20260926 04:07:23.022454 22895 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:23.022985 22895 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:23.028630 22895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19c9b498dbd946148906539e0a93b20d: Generated new TSK 0
I20260926 04:07:23.028800 22895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:23.032553 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22813
2026-09-26T04:07:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:23 dist-test-slave-j8c1 krb5kdc[22912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:23 dist-test-slave-j8c1 krb5kdc[22912](info): set up 2 sockets
Sep 26 04:07:23 dist-test-slave-j8c1 krb5kdc[22912](info): commencing operation
krb5kdc: starting...
W20260926 04:07:24.090822 802 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:24 dist-test-slave-j8c1 krb5kdc[22912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:24.219004 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39849
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:39849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:24.305327 22925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:24.305548 22925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:24.305572 22925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:24.306852 22925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:24.306896 22925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:24.306911 22925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:24.306926 22925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:24.306969 22925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:24.308473 22925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:39849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:39849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:24.308722 22925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:24.308954 22925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:24.313786 22933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:24.313807 22930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:24.313948 22925 server_base.cc:1034] running on GCE node
W20260926 04:07:24.313848 22931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:24.314347 22925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:24.314675 22925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:24.315893 22925 hybrid_clock.cc:648] HybridClock initialized: now 1790395644315873 us; error 35 us; skew 500 ppm
Sep 26 04:07:24 dist-test-slave-j8c1 krb5kdc[22912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:24.323652 22925 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:24.324193 22925 webserver.cc:460] Webserver started at http://127.0.200.190:43503/ using document root <none> and password file <none>
I20260926 04:07:24.324456 22925 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:24.324528 22925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:24.324684 22925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:24.325786 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "2bf61c2ea43c44a4b165f346f50f80f9"
format_stamp: "Formatted at 2026-09-26 04:07:24 on dist-test-slave-j8c1"
server_key: "40182e485c313159581b972af2334559"
server_key_iv: "735d0b130e37077f9a3faa08fd24083d"
server_key_version: "encryptionkey@0"
I20260926 04:07:24.326171 22925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "2bf61c2ea43c44a4b165f346f50f80f9"
format_stamp: "Formatted at 2026-09-26 04:07:24 on dist-test-slave-j8c1"
server_key: "40182e485c313159581b972af2334559"
server_key_iv: "735d0b130e37077f9a3faa08fd24083d"
server_key_version: "encryptionkey@0"
I20260926 04:07:24.327708 22925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:24.328553 22940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:24.328755 22925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:24.328910 22925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "2bf61c2ea43c44a4b165f346f50f80f9"
format_stamp: "Formatted at 2026-09-26 04:07:24 on dist-test-slave-j8c1"
server_key: "40182e485c313159581b972af2334559"
server_key_iv: "735d0b130e37077f9a3faa08fd24083d"
server_key_version: "encryptionkey@0"
I20260926 04:07:24.328991 22925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:24.332530 22925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:24.335028 22925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:24.335206 22925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:24.339265 22925 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:39849
I20260926 04:07:24.339295 22992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:39849 every 8 connection(s)
I20260926 04:07:24.339648 22925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:24.340741 22993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:24.343423 22993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9: Bootstrap starting.
I20260926 04:07:24.344056 22993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:24.344218 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 22925
I20260926 04:07:24.344350 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:24.344455 22993 log.cc:824] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:24.344588 802 external_mini_cluster.cc:1428] Setting key 6a320462761b1b737231bd00d8196f73
I20260926 04:07:24.345327 22993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9: No bootstrap required, opened a new log
I20260926 04:07:24.347407 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:24.345142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55953 (local address 127.0.200.190:39849)
0926 04:07:24.345339 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:24.345357 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:24.345395 (+ 38us) server_negotiation.cc:407] Connection header received
0926 04:07:24.345877 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:24.345884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:24.345910 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:24.346084 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:24.346982 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:24.346986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:24.346986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:24.346991 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:24.347040 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:24.347051 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:24.347151 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:24.347192 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 04:07:24.347233 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260926 04:07:24.348448 22993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } }
I20260926 04:07:24.348671 22993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 04:07:24.348713 22993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf61c2ea43c44a4b165f346f50f80f9, State: Initialized, Role: FOLLOWER
I20260926 04:07:24.348737 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:24.348845 22993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } }
I20260926 04:07:24.348937 22993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 04:07:24.348989 22993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 04:07:24.349045 22993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:24.349674 22993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } }
I20260926 04:07:24.349808 22993 leader_election.cc:302] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bf61c2ea43c44a4b165f346f50f80f9; no voters:
I20260926 04:07:24.349987 22993 leader_election.cc:288] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 04:07:24.350015 23002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:24.350173 23002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [term 1 LEADER]: Becoming Leader. State: Replica: 2bf61c2ea43c44a4b165f346f50f80f9, State: Running, Role: LEADER
I20260926 04:07:24.350313 23002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } }
I20260926 04:07:24.350318 22993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:24.350862 23003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } } }
I20260926 04:07:24.350911 23004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf61c2ea43c44a4b165f346f50f80f9. Latest consensus state: current_term: 1 leader_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf61c2ea43c44a4b165f346f50f80f9" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 39849 } } }
I20260926 04:07:24.351044 23004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:24.350977 23003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:24.351403 23012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:24.351940 23012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:24.352167 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:24.350202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59003 (local address 127.0.200.190:39849)
0926 04:07:24.350239 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:24.350241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:24.351025 (+ 784us) server_negotiation.cc:407] Connection header received
0926 04:07:24.351051 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:24.351052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:24.351058 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:24.351126 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:24.351893 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:24.351895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:24.351895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 04:07:24.351900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:24.351917 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 04:07:24.351921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:24.351964 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:24.352078 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 04:07:24.352094 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:07:24.357800 23012 catalog_manager.cc:1347] Generated new cluster ID: 4faea33b20804352b6b684cedf456e3a
I20260926 04:07:24.357887 23012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:24.366971 23012 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:24.367165 23012 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:24.379869 23012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bf61c2ea43c44a4b165f346f50f80f9: Generated new TSK 0
I20260926 04:07:24.380100 23012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:24.383076 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 22925
2026-09-26T04:07:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1350 ms)
[----------] 18 tests from AuthTokenIssuingTest (24143 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:24 dist-test-slave-j8c1 krb5kdc[23024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:24 dist-test-slave-j8c1 krb5kdc[23024](info): set up 2 sockets
Sep 26 04:07:24 dist-test-slave-j8c1 krb5kdc[23024](info): commencing operation
krb5kdc: starting...
W20260926 04:07:25.431166 802 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:25.553869 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36747
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:25.634500 23037 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:25.634793 23037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:25.634835 23037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:25.634891 23037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:25.635996 23037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:25.636061 23037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:25.636086 23037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:25.636116 23037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:25.636138 23037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:25.637606 23037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:36747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:25.637876 23037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:25.638095 23037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:25.642875 23043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:25.642884 23045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:25.643100 23037 server_base.cc:1034] running on GCE node
W20260926 04:07:25.642880 23042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:25.643386 23037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:25.643682 23037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:25.644884 23037 hybrid_clock.cc:648] HybridClock initialized: now 1790395645644869 us; error 26 us; skew 500 ppm
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:25.647257 23037 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:25.647701 23037 webserver.cc:460] Webserver started at http://127.0.200.190:40909/ using document root <none> and password file <none>
I20260926 04:07:25.647881 23037 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:25.647919 23037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:25.648005 23037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:25.649044 23037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "c55b892c5cec471c92e9341bcde14841"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "e31305892c1165033f7c27698bf8caba"
server_key_iv: "1cdf704e2b06e386d3b72402366fd651"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.649346 23037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "c55b892c5cec471c92e9341bcde14841"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "e31305892c1165033f7c27698bf8caba"
server_key_iv: "1cdf704e2b06e386d3b72402366fd651"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.650679 23037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:25.651419 23052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:25.651609 23037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:25.651705 23037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "c55b892c5cec471c92e9341bcde14841"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "e31305892c1165033f7c27698bf8caba"
server_key_iv: "1cdf704e2b06e386d3b72402366fd651"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.651762 23037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:25.668931 23037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:25.671083 23037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:25.671195 23037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:25.675112 23037 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:36747
I20260926 04:07:25.675221 23104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:36747 every 8 connection(s)
I20260926 04:07:25.675510 23037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:25.676367 23105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:25.677857 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:25.678972 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23037
I20260926 04:07:25.679054 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:25.679248 802 external_mini_cluster.cc:1428] Setting key c9392fa3063b4f2915560d43a1d2e090
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.189@KRBTEST.COM; defaulting to no policy
I20260926 04:07:25.688071 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.678605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.0.200.190:36747)
0926 04:07:25.678789 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.678795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.678826 (+ 31us) server_negotiation.cc:407] Connection header received
0926 04:07:25.679452 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.679454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.679473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.679576 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.680105 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.681442 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.682291 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.682684 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.682808 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.684836 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.684843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.684843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.684850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.686415 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.686909 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.686911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.686912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.686993 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.687223 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.687224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.687224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.687429 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:25.687522 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:25.687721 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 04:07:25.687837 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260926 04:07:25.688078 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.678530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36747 (local address 127.0.0.1:54182)
0926 04:07:25.678712 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.678753 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.679313 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.679579 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.679581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.679862 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.680008 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.680013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.681517 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.681518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.682204 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.682206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.682793 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.682794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.682859 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.683399 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.683407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:25.684735 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:25.686512 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.686519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.686521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.686811 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.687062 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.687063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.687063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.687156 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.687501 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:25.687506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:25.687563 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 04:07:25.687688 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 04:07:25.687892 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":78,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.189@KRBTEST.COM" created.
I20260926 04:07:25.688872 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:25.689236 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.689122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33136)
0926 04:07:25.689176 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.689216 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 04:07:25.689393 23053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
I20260926 04:07:25.689806 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.689723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33148)
0926 04:07:25.689767 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.689790 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 04:07:25.689939 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:40993: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
I20260926 04:07:25.703305 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:40993
--webserver_interface=127.0.200.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:07:25.739192 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } attempt: 1
I20260926 04:07:25.739737 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.739579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33150)
0926 04:07:25.739657 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.739701 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 04:07:25.740036 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.739940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33164)
0926 04:07:25.739989 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.740015 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 04:07:25.740214 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:40993: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
W20260926 04:07:25.798203 23112 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:25.798410 23112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:25.798429 23112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:25.798448 23112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:25.799587 23112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:25.799630 23112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:25.799644 23112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:25.799657 23112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:25.799669 23112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:25.801064 23112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.200.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:25.801249 23112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:25.801415 23112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:25.806059 23118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:25.806061 23120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:25.806089 23117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:25.806404 23112 server_base.cc:1034] running on GCE node
I20260926 04:07:25.806752 23112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:25.807052 23112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:25.808202 23112 hybrid_clock.cc:648] HybridClock initialized: now 1790395645808190 us; error 32 us; skew 500 ppm
I20260926 04:07:25.808362 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } attempt: 2
I20260926 04:07:25.808776 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.808633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33178)
0926 04:07:25.808699 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.808733 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 04:07:25.809127 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.809006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33188)
0926 04:07:25.809064 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.809097 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 04:07:25.809227 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:40993: Network error: Client connection negotiation failed: client connection to 127.0.200.189:40993: connect: Connection refused (error 111)
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:25.810941 23112 init.cc:377] Logged in from keytab as kudu/127.0.200.189@KRBTEST.COM (short username kudu)
I20260926 04:07:25.811403 23112 webserver.cc:460] Webserver started at http://127.0.200.189:39129/ using document root <none> and password file <none>
I20260926 04:07:25.811626 23112 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:25.811690 23112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:25.811836 23112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:25.812860 23112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data/instance:
uuid: "e5876049efb24248b0255c2cc5eeb3b8"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "b7b1b950523787579361def7621b7e55"
server_key_iv: "3c375941b17d0cf6f9436c5d371cc8fc"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.813200 23112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal/instance:
uuid: "e5876049efb24248b0255c2cc5eeb3b8"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "b7b1b950523787579361def7621b7e55"
server_key_iv: "3c375941b17d0cf6f9436c5d371cc8fc"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.814497 23112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:25.815228 23127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:25.815380 23112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 04:07:25.815476 23112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "e5876049efb24248b0255c2cc5eeb3b8"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "b7b1b950523787579361def7621b7e55"
server_key_iv: "3c375941b17d0cf6f9436c5d371cc8fc"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.815560 23112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:25.829416 23112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:25.831605 23112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:25.831759 23112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:25.835783 23112 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.189:40993
I20260926 04:07:25.835815 23179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.189:40993 every 8 connection(s)
I20260926 04:07:25.836181 23112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
I20260926 04:07:25.837133 23180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:25.837913 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23112
I20260926 04:07:25.838043 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-1/wal/instance
I20260926 04:07:25.838215 802 external_mini_cluster.cc:1428] Setting key 9d9b937a781dad7db94bf4dd4831547f
I20260926 04:07:25.838899 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.200.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.188@KRBTEST.COM" created.
I20260926 04:07:25.848945 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.839685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.0.200.190:36747)
0926 04:07:25.839771 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.839774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.839997 (+ 223us) server_negotiation.cc:407] Connection header received
0926 04:07:25.840455 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.840456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.840473 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.840553 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.841198 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.841758 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.843251 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.843467 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.843531 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.845896 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.845897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.845898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.845906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.847496 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.847925 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.847926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.847927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.848026 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.848281 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.848282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.848282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.848397 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:25.848453 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:25.848766 (+ 313us) server_negotiation.cc:299] Negotiation successful
0926 04:07:25.848851 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 04:07:25.849131 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.839670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36747 (local address 127.0.0.1:54184)
0926 04:07:25.839861 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.839902 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.840361 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.840617 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.840620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.840859 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.841094 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.841098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.841849 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.841850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.843154 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.843157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.843579 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.843580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.843741 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.844243 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.844260 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:25.845795 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:25.847571 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.847577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.847579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.847808 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.848103 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.848103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.848103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.848217 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.848457 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:25.848461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:25.848619 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 04:07:25.848740 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 04:07:25.848894 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260926 04:07:25.850628 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:25.858628 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.850894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33198)
0926 04:07:25.850941 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.850954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.850995 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.851327 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.851328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.851489 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.851548 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.851550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.852492 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.852493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.853044 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.853045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.853548 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.853548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.853634 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.854117 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.854121 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:25.855729 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:25.857549 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.857550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.857550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.857723 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.857918 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.857919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.857919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.857975 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.858307 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:25.858308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:25.858355 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 04:07:25.858410 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 04:07:25.858491 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 04:07:25.858799 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.850959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.0.200.189:40993)
0926 04:07:25.851145 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.851150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.851174 (+ 24us) server_negotiation.cc:407] Connection header received
0926 04:07:25.851201 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.851203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.851220 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.851314 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.851633 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.852373 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.853134 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.853463 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.853553 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.855830 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.855837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.855838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.855845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.857470 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.857799 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.857801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.857802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.857857 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.858042 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.858043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.858043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.858234 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:25.858305 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:25.858442 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 04:07:25.858607 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260926 04:07:25.859426 23180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:25.859843 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.859722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51654)
0926 04:07:25.859783 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.859825 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 04:07:25.859977 23130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
I20260926 04:07:25.860371 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.860291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51662)
0926 04:07:25.860334 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.860357 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260926 04:07:25.860486 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38433: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
I20260926 04:07:25.865592 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:38433
--webserver_interface=127.0.200.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:07:25.900405 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } attempt: 3
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:25.907974 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.900793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33210)
0926 04:07:25.900856 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.900871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.900940 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.901095 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.901095 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.901280 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.901345 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.901347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.902101 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.902102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.902707 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.902710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.903150 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.903151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.903236 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.903755 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.903760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:25.905200 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:25.906991 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.906992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.906992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.907170 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.907377 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.907378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.907378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.907433 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.907693 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:25.907693 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:25.907765 (+ 72us) client_negotiation.cc:769] Sending connection context
0926 04:07:25.907821 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 04:07:25.907900 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 04:07:25.908035 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.900892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.0.200.189:40993)
0926 04:07:25.900968 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.900970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.900983 (+ 13us) server_negotiation.cc:407] Connection header received
0926 04:07:25.901011 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.901012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.901027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.901089 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.901430 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.902006 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.902807 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.903032 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.903150 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.905290 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.905291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.905292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.905300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.906892 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.907247 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.907248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.907248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.907309 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.907494 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.907495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.907496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.907631 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:25.907697 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:25.907863 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 04:07:25.907944 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 04:07:25.908468 23105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:25.908844 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.908711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51664)
0926 04:07:25.908771 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.908814 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 04:07:25.909287 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.909189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51666)
0926 04:07:25.909237 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.909265 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260926 04:07:25.909406 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38433: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
I20260926 04:07:25.909716 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } attempt: 1
I20260926 04:07:25.910089 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.909981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51668)
0926 04:07:25.910035 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.910067 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:07:25.910382 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.910294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51672)
0926 04:07:25.910337 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.910359 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260926 04:07:25.910461 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38433: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
I20260926 04:07:25.940595 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } attempt: 1
I20260926 04:07:25.941131 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.940972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51686)
0926 04:07:25.941066 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.941100 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 04:07:25.941565 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.941449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51690)
0926 04:07:25.941511 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.941542 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 04:07:25.941697 23105 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38433: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
W20260926 04:07:25.948490 23187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:25.948724 23187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:25.948740 23187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:25.948761 23187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:25.949930 23187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:25.949975 23187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:25.949988 23187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:25.950002 23187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:25.950053 23187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:25.951680 23187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:36747,127.0.200.189:40993,127.0.200.188:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.200.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:25.951942 23187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:25.952178 23187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:25.956983 23193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:25.957000 23187 server_base.cc:1034] running on GCE node
W20260926 04:07:25.956979 23192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:25.956979 23195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:25.957581 23187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:25.957885 23187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:25.959043 23187 hybrid_clock.cc:648] HybridClock initialized: now 1790395645959033 us; error 31 us; skew 500 ppm
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:25.961632 23187 init.cc:377] Logged in from keytab as kudu/127.0.200.188@KRBTEST.COM (short username kudu)
I20260926 04:07:25.962073 23187 webserver.cc:460] Webserver started at http://127.0.200.188:36597/ using document root <none> and password file <none>
I20260926 04:07:25.962311 23187 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:25.962378 23187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:25.962530 23187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:25.963657 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data/instance:
uuid: "e7e2031995f84ffab0d249110b0f275d"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "4ef7a5a09193dfd3be51ad7066a84f11"
server_key_iv: "ccee0c3ae45d950073ded2364e6b4966"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.964008 23187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal/instance:
uuid: "e7e2031995f84ffab0d249110b0f275d"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "4ef7a5a09193dfd3be51ad7066a84f11"
server_key_iv: "ccee0c3ae45d950073ded2364e6b4966"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.965348 23187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 04:07:25.966102 23202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:25.966296 23187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:25.966401 23187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "e7e2031995f84ffab0d249110b0f275d"
format_stamp: "Formatted at 2026-09-26 04:07:25 on dist-test-slave-j8c1"
server_key: "4ef7a5a09193dfd3be51ad7066a84f11"
server_key_iv: "ccee0c3ae45d950073ded2364e6b4966"
server_key_version: "encryptionkey@0"
I20260926 04:07:25.966485 23187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:25.976336 23187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:25.978436 23187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:25.978664 23187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:25.978637 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } attempt: 2
I20260926 04:07:25.979121 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.978959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51698)
0926 04:07:25.979032 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.979079 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 04:07:25.979509 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.979401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51706)
0926 04:07:25.979452 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.979478 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 04:07:25.979636 23180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38433: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38433: connect: Connection refused (error 111)
I20260926 04:07:25.982908 23187 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.188:38433
I20260926 04:07:25.982995 23254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.188:38433 every 8 connection(s)
I20260926 04:07:25.983330 23187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
I20260926 04:07:25.984377 23255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:25.986021 23255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:25.990232 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23187
I20260926 04:07:25.990367 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790395371830964-802-0/minicluster-data/master-2/wal/instance
I20260926 04:07:25.990633 802 external_mini_cluster.cc:1428] Setting key 64dd8f8abbb9f5f9947b875a4c82653b
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:25.995930 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.986820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.0.200.190:36747)
0926 04:07:25.986866 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.986868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.987144 (+ 276us) server_negotiation.cc:407] Connection header received
0926 04:07:25.987669 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.987670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.987684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.987738 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.988391 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.988881 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.989874 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.990071 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.990155 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.992700 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.992701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.992701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.992709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.994305 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.994847 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.994848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.994849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.994915 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.995282 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.995282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.995283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.995393 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:25.995443 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:25.995798 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 04:07:25.995862 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:25.996237 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.986840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:36747 (local address 127.0.0.1:54186)
0926 04:07:25.987013 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.987059 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.987547 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.987841 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.987845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.988099 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.988288 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.988292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.988974 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.988974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.989768 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.989771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.990177 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.990178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.990334 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.990944 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.990953 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:25.992603 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:25.994392 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.994398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.994399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.994772 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.995007 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:25.995008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:25.995008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:25.995193 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:25.995463 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:25.995470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:25.995647 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 04:07:25.995788 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 04:07:25.995962 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:25.996803 23255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:26.000527 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.991055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.0.200.188:38433)
0926 04:07:25.991251 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.991256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.991294 (+ 38us) server_negotiation.cc:407] Connection header received
0926 04:07:25.991339 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.991342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.991362 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.991429 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.992201 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.992944 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.993867 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.994381 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.994442 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.997376 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:25.997383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:25.997383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:25.997390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:25.999092 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.999550 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.999553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.999554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:25.999639 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:25.999848 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:25.999849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:25.999850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.000018 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.000177 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.000295 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.000380 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
Sep 26 04:07:25 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:26.003896 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.997074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:40993 (local address 127.0.0.1:33222)
0926 04:07:25.997128 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:25.997140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:25.997186 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:25.997302 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:25.997303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:25.997440 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:25.997486 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.997487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.998114 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.998115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.998668 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:25.998670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.999084 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.999085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:25.999155 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:25.999568 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:25.999572 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:26.000985 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:26.002843 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.002844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.002845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.003010 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.003231 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.003231 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.003232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.003288 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.003555 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:26.003556 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:26.003625 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 04:07:26.003710 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 04:07:26.003803 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:07:26.003906 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:25.997156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.0.200.189:40993)
0926 04:07:25.997199 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:25.997201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:25.997209 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:07:25.997246 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:25.997246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:25.997256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:25.997301 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:25.997572 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.998033 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.998763 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:25.998992 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:25.999085 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.001089 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.001090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.001091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.001106 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.002755 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.003101 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.003102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.003103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.003156 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.003367 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.003368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.003368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.003480 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.003562 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.003752 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.003819 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260926 04:07:26.004451 23255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:07:26 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
Sep 26 04:07:26 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:26.009809 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.001401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.0.200.190:36747)
0926 04:07:26.001439 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.001441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.001657 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:07:26.001682 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.001683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.001695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.001762 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.002213 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.002728 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.003394 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.003590 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.003670 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.005955 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.005956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.005957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.005964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.008135 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.008513 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.008514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.008514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.008567 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.009068 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.009068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.009069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.009178 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.009289 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.009679 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.009746 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260926 04:07:26.011179 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.004762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51716)
0926 04:07:26.004802 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:26.004811 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:26.004846 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:26.004964 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:26.004965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:26.005097 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:26.005122 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.005123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.005789 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.005790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.006251 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.006253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.006662 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.006663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.006710 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.007074 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:26.007078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:26.008331 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:26.010197 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.010198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.010198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.010382 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.010575 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.010576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.010576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.010629 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.010869 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:26.010870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:26.010905 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 04:07:26.010972 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 04:07:26.011068 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 04:07:26.011234 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.004810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.0.200.188:38433)
0926 04:07:26.004845 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.004847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.004854 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:07:26.004906 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.004907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.004916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.004962 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.005191 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.005715 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.006341 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.006609 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.006666 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.008423 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.008424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.008424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.008430 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.010103 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.010431 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.010432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.010433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.010491 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.010700 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.010701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.010701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.010806 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.010878 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.010998 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.011112 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":20}
I20260926 04:07:26.013377 23255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: Bootstrap starting.
I20260926 04:07:26.014217 23255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:26.014544 23255 log.cc:824] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: Log is configured to *not* fsync() on all Append() calls
Sep 26 04:07:26 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:26.015659 23255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: No bootstrap required, opened a new log
I20260926 04:07:26.016865 23105 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } attempt: 2
I20260926 04:07:26.018115 23255 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.018319 23255 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7e2031995f84ffab0d249110b0f275d, State: Initialized, Role: FOLLOWER
I20260926 04:07:26.018455 23255 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.018833 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.010531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.0.200.189:40993)
0926 04:07:26.010590 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.010592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.010600 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:07:26.010644 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.010645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.010653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.010724 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.011179 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.011794 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.012570 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.012776 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.012860 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.015366 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.015367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.015368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.015374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.017455 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.017824 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.017825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.017826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.017887 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.018125 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.018126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.018127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.018259 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.018444 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.018647 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.018736 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 04:07:26.018826 23255 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:26.018812 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.018921 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.020609 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260926 04:07:26.020807 23272 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:26.020855 23272 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 04:07:26 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:26.021605 23079 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c55b892c5cec471c92e9341bcde14841"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54200
I20260926 04:07:26.024770 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.017136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51722)
0926 04:07:26.017195 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:26.017206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:26.017254 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:26.017392 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:26.017393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:26.017558 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:26.017607 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.017609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.018605 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.018606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.019405 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.019409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.019906 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.019908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.019966 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.020389 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:26.020393 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:26.021676 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:26.023624 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.023625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.023626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.023871 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.024086 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.024087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.024087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.024152 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.024439 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:26.024440 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:26.024495 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 04:07:26.024571 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 04:07:26.024671 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260926 04:07:26.024783 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.017196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.0.200.188:38433)
0926 04:07:26.017232 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.017234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.017242 (+ 8us) server_negotiation.cc:407] Connection header received
0926 04:07:26.017312 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.017313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.017322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.017388 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.017695 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.018479 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.019498 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.019818 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.021512 (+ 1694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.021770 (+ 258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.021771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.021772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.021777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.023480 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.023955 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.023956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.023956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.024015 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.024224 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.024225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.024226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.024382 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.024449 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.024607 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.024697 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:26.022426 23187 master_runner.cc:186] Error getting master registration for 127.0.200.190:36747: OK, instance_id { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" instance_seqno: 1790395645673950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:07:26.024863 23187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:07:26.026695 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841: Bootstrap starting.
I20260926 04:07:26.027431 23105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:26.027750 23105 log.cc:824] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:26.028595 23105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841: No bootstrap required, opened a new log
I20260926 04:07:26.030014 23105 raft_consensus.cc:348] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.030215 23105 raft_consensus.cc:729] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55b892c5cec471c92e9341bcde14841, State: Initialized, Role: FOLLOWER
I20260926 04:07:26.030314 23105 consensus_queue.cc:260] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.030637 23275 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.030727 23079 raft_consensus.cc:482] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 04:07:26.030709 23105 sys_catalog.cc:564] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:26.030781 23275 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.030818 23079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:26.031519 23079 raft_consensus.cc:504] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.031863 23079 leader_election.cc:288] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [CANDIDATE]: Term 1 election: Requested vote from peers e5876049efb24248b0255c2cc5eeb3b8 (127.0.200.189:40993), e7e2031995f84ffab0d249110b0f275d (127.0.200.188:38433)
I20260926 04:07:26.032399 23229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c55b892c5cec471c92e9341bcde14841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e7e2031995f84ffab0d249110b0f275d"
I20260926 04:07:26.032595 23229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:26.033366 23229 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b892c5cec471c92e9341bcde14841 in term 1.
I20260926 04:07:26.034029 23055 leader_election.cc:302] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c55b892c5cec471c92e9341bcde14841, e7e2031995f84ffab0d249110b0f275d; no voters:
I20260926 04:07:26.034165 23275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:26.034302 23275 raft_consensus.cc:686] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [term 1 LEADER]: Becoming Leader. State: Replica: c55b892c5cec471c92e9341bcde14841, State: Running, Role: LEADER
I20260926 04:07:26.034382 23275 consensus_queue.cc:237] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.034868 23275 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c55b892c5cec471c92e9341bcde14841. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.034953 23275 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:26.035296 23293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:26.033397 23037 master_runner.cc:186] Error getting master registration for 127.0.200.189:40993: OK, instance_id { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" instance_seqno: 1790395645834644 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:07:26.035866 23037 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:07:26.035797 23293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:26.037954 23229 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [term 1 FOLLOWER]: Refusing update from remote peer c55b892c5cec471c92e9341bcde14841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:26.038365 23291 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:07:26.039712 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c55b892c5cec471c92e9341bcde14841. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.039801 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.041333 23275 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.041417 23275 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:26.041455 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.041525 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.041929 23298 mvcc.cc:205] Tried to move back new op lower bound from 7333460566168608768 to 7333460566158045184. Current Snapshot: MvccSnapshot[applied={T|T < 7333460566168608768}]
I20260926 04:07:26.042702 23299 mvcc.cc:205] Tried to move back new op lower bound from 7333460566168608768 to 7333460566158045184. Current Snapshot: MvccSnapshot[applied={T|T < 7333460566168608768}]
I20260926 04:07:26.043494 23293 catalog_manager.cc:1347] Generated new cluster ID: 1b3308184df24221849d255bc1b93d5a
I20260926 04:07:26.043570 23293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:26.058145 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.036629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.0.200.188:38433)
0926 04:07:26.040232 (+ 3603us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.040234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.043355 (+ 3121us) server_negotiation.cc:407] Connection header received
0926 04:07:26.044283 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.044284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.044294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.044320 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.051415 (+ 7095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.052150 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.053176 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.053459 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.053494 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.055050 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.055051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.055052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.055058 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.056967 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.057388 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.057390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.057390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.057447 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.057607 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.057608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.057608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.057726 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.057798 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.057971 (+ 173us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.058048 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3579}
I20260926 04:07:26.061941 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.036453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.0.200.190:36747)
0926 04:07:26.036489 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.036491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.043252 (+ 6761us) server_negotiation.cc:407] Connection header received
0926 04:07:26.043505 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.043506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.043540 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.043572 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.053169 (+ 9597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.053926 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.055811 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.056121 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.056183 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.058451 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.058453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.058453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.058462 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.060134 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.060798 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.060799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.060799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.060857 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.061164 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.061165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.061179 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.061330 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.061415 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.061746 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.061831 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260926 04:07:26.062220 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.036203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.0.200.189:40993)
0926 04:07:26.036259 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.036262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.040366 (+ 4104us) server_negotiation.cc:407] Connection header received
0926 04:07:26.040480 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.040481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.040493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.040526 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.041375 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.042084 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.050643 (+ 8559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.050971 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.051017 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.059383 (+ 8366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.059386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.059386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.059394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.060969 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.061361 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.061362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.061362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.061419 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.061606 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.061607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.061607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.061747 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.061837 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.062039 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.062126 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:26.068838 23293 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:26.069656 23293 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:26.070735 23180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } attempt: 3
Sep 26 04:07:26 dist-test-slave-j8c1 krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:26.078637 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.070997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38433 (local address 127.0.0.1:51740)
0926 04:07:26.071045 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:26.071057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:26.071095 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:26.071283 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:26.071284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:26.071468 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:26.071513 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.071515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.072376 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.072377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.073149 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:26.073152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.073656 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.073657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:26.073705 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.074193 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:26.074198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:26.075674 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:26.077411 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.077412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.077412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.077640 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.077885 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:26.077886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:26.077886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:26.077952 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:26.078277 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:26.078277 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:26.078324 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 04:07:26.078386 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 04:07:26.078490 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 04:07:26.078984 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:26.071042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.0.200.188:38433)
0926 04:07:26.071082 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.071085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.071094 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:26.071159 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.071159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.071168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.071232 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.071584 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.072292 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.073242 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:26.073536 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:26.073592 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:26.075768 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:26.075769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:26.075769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:26.075776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:26.077313 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.077733 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.077734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.077735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.077799 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:26.078021 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:26.078022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:26.078022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:26.078185 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:26.078249 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:26.078814 (+ 565us) server_negotiation.cc:299] Negotiation successful
0926 04:07:26.078894 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:07:26.079421 23293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841: Generated new TSK 0
I20260926 04:07:26.079885 23293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 04:07:26.080785 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: Bootstrap starting.
I20260926 04:07:26.081590 23180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:26.081898 23180 log.cc:824] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:26.082840 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: No bootstrap required, opened a new log
I20260926 04:07:26.085078 23180 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.085237 23180 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5876049efb24248b0255c2cc5eeb3b8, State: Initialized, Role: FOLLOWER
I20260926 04:07:26.085321 23180 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } }
I20260926 04:07:26.085700 23180 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:26.085755 23154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c55b892c5cec471c92e9341bcde14841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e5876049efb24248b0255c2cc5eeb3b8"
I20260926 04:07:26.085860 23154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:26.086781 23154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b892c5cec471c92e9341bcde14841 in term 1.
I20260926 04:07:26.086979 23153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [term 1 FOLLOWER]: Refusing update from remote peer c55b892c5cec471c92e9341bcde14841: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:26.085696 23304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.087342 23304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.087636 23291 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:07:26.089331 23304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c55b892c5cec471c92e9341bcde14841. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.089455 23304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:26.089538 23304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.089673 23304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 04:07:26.090494 23315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:26.090567 23315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:26.095182 23317 mvcc.cc:205] Tried to move back new op lower bound from 7333460566333415424 to 7333460566158045184. Current Snapshot: MvccSnapshot[applied={T|T < 7333460566168608768}]
W20260926 04:07:26.109133 23108 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.107834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.0.200.190:36747)
0926 04:07:26.107879 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.107882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.108458 (+ 576us) server_negotiation.cc:407] Connection header received
0926 04:07:26.108561 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.108563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.108574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.108605 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.109060 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54218: BlockingRecv error: recv got EOF from 127.0.0.1:54218 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 04:07:26.110139 23259 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.108181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.0.200.188:38433)
0926 04:07:26.108220 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.108222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.109573 (+ 1351us) server_negotiation.cc:407] Connection header received
0926 04:07:26.109725 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.109726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.109737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.109766 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.110076 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51744: BlockingRecv error: recv got EOF from 127.0.0.1:51744 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 04:07:26.110958 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.109407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.0.200.189:40993)
0926 04:07:26.109444 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.109446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.110331 (+ 885us) server_negotiation.cc:407] Connection header received
0926 04:07:26.110486 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.110487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.110497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.110526 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.110902 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33250: BlockingRecv error: recv got EOF from 127.0.0.1:33250 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 04:07:26.113252 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.112532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.0.200.189:40993)
0926 04:07:26.112565 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.112567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.112748 (+ 181us) server_negotiation.cc:407] Connection header received
0926 04:07:26.112854 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.112855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.112873 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.112902 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.113204 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33260: BlockingRecv error: recv got EOF from 127.0.0.1:33260 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 04:07:26.116225 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.115480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.0.200.189:40993)
0926 04:07:26.115515 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.115517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.115747 (+ 230us) server_negotiation.cc:407] Connection header received
0926 04:07:26.115844 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.115845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.115855 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.115884 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.116181 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33274: BlockingRecv error: recv got EOF from 127.0.0.1:33274 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260926 04:07:26.120239 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.119465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.0.200.189:40993)
0926 04:07:26.119499 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.119501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.119748 (+ 247us) server_negotiation.cc:407] Connection header received
0926 04:07:26.119845 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.119846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.119856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.119885 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.120193 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33286: BlockingRecv error: recv got EOF from 127.0.0.1:33286 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260926 04:07:26.126250 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.125445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.0.200.189:40993)
0926 04:07:26.125482 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.125484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.125707 (+ 223us) server_negotiation.cc:407] Connection header received
0926 04:07:26.125815 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.125816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.125828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.125857 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.126199 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33294: BlockingRecv error: recv got EOF from 127.0.0.1:33294 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 04:07:26.136392 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.135544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.0.200.189:40993)
0926 04:07:26.135585 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.135587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.135830 (+ 243us) server_negotiation.cc:407] Connection header received
0926 04:07:26.135933 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.135935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.135947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.135977 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.136332 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33308: BlockingRecv error: recv got EOF from 127.0.0.1:33308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260926 04:07:26.154867 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.153854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.0.200.189:40993)
0926 04:07:26.153901 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.153903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.154159 (+ 256us) server_negotiation.cc:407] Connection header received
0926 04:07:26.154270 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.154271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.154284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.154334 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.154808 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33316: BlockingRecv error: recv got EOF from 127.0.0.1:33316 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260926 04:07:26.189262 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.188328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.0.200.189:40993)
0926 04:07:26.188374 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.188376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.188630 (+ 254us) server_negotiation.cc:407] Connection header received
0926 04:07:26.188754 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.188756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.188768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.188800 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.189202 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33328: BlockingRecv error: recv got EOF from 127.0.0.1:33328 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 04:07:26.255529 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.254822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.0.200.189:40993)
0926 04:07:26.254891 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.254893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.254976 (+ 83us) server_negotiation.cc:407] Connection header received
0926 04:07:26.255015 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.255017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.255052 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.255132 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.255465 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33342: BlockingRecv error: recv got EOF from 127.0.0.1:33342 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260926 04:07:26.385485 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.384680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.0.200.189:40993)
0926 04:07:26.384757 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.384759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.384912 (+ 153us) server_negotiation.cc:407] Connection header received
0926 04:07:26.384953 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.384954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.384966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.385017 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.385417 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39552: BlockingRecv error: recv got EOF from 127.0.0.1:39552 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 04:07:26.569823 23291 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c55b892c5cec471c92e9341bcde14841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b892c5cec471c92e9341bcde14841" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 36747 } } peers { permanent_uuid: "e5876049efb24248b0255c2cc5eeb3b8" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 40993 } } peers { permanent_uuid: "e7e2031995f84ffab0d249110b0f275d" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38433 } } }
I20260926 04:07:26.570011 23291 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b892c5cec471c92e9341bcde14841 [sys.catalog]: This master's current role is: LEADER
W20260926 04:07:26.643498 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:26.642739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.0.200.189:40993)
0926 04:07:26.642815 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:26.642817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:26.642909 (+ 92us) server_negotiation.cc:407] Connection header received
0926 04:07:26.642986 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:26.642987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:26.643000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:26.643096 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:26.643424 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39564: BlockingRecv error: recv got EOF from 127.0.0.1:39564 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 04:07:27.021281 23272 catalog_manager.cc:1259] Loaded cluster ID: 1b3308184df24221849d255bc1b93d5a
I20260926 04:07:27.021358 23272 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: loading cluster ID for follower catalog manager: success
I20260926 04:07:27.022157 23272 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: acquiring CA information for follower catalog manager: success
I20260926 04:07:27.022450 23272 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e7e2031995f84ffab0d249110b0f275d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:07:27.090989 23315 catalog_manager.cc:1259] Loaded cluster ID: 1b3308184df24221849d255bc1b93d5a
I20260926 04:07:27.091068 23315 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: loading cluster ID for follower catalog manager: success
I20260926 04:07:27.091926 23315 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: acquiring CA information for follower catalog manager: success
I20260926 04:07:27.092237 23315 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e5876049efb24248b0255c2cc5eeb3b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:07:27.159097 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:27.156639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.0.200.189:40993)
0926 04:07:27.156763 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:27.156765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:27.156775 (+ 10us) server_negotiation.cc:407] Connection header received
0926 04:07:27.156820 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:27.156821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:27.156868 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:27.156932 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:27.157065 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.157576 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.158337 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.158602 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.158667 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:27.158803 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:27.158917 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:27.158977 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:27.159009 (+ 32us) server_negotiation.cc:299] Negotiation successful
0926 04:07:27.159029 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260926 04:07:27.163223 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:27.160949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.0.200.188:38433)
0926 04:07:27.161051 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:27.161052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:27.161101 (+ 49us) server_negotiation.cc:407] Connection header received
0926 04:07:27.161159 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:27.161160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:27.161193 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:27.161260 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:27.161326 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.161846 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.162596 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.162789 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.162840 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:27.162950 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:27.163045 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:27.163100 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:27.163141 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 04:07:27.163158 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
I20260926 04:07:27.167863 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:27.165060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.0.200.189:40993)
0926 04:07:27.165109 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:27.165111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:27.165287 (+ 176us) server_negotiation.cc:407] Connection header received
0926 04:07:27.165389 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:27.165391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:27.165403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:27.165435 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:27.165640 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.166130 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167088 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.167384 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167424 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:27.167597 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:27.167631 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:27.167679 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:27.167743 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 04:07:27.167765 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260926 04:07:27.168288 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:27.165418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.0.200.188:38433)
0926 04:07:27.165457 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:27.165459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:27.165466 (+ 7us) server_negotiation.cc:407] Connection header received
0926 04:07:27.165488 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:27.165489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:27.165496 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:27.165558 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:27.165707 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.166433 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167418 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.167702 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167758 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:27.167946 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:27.167979 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:27.168009 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:27.168195 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 04:07:27.168221 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260926 04:07:27.168284 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:27.165063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.0.200.190:36747)
0926 04:07:27.165407 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:27.165410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:27.165419 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:27.165466 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:27.165466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:27.165521 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:27.165628 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:27.165769 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.166508 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167250 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:27.167518 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:27.167724 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:27.167797 (+ 73us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:27.167912 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:27.168130 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:27.168158 (+ 28us) server_negotiation.cc:299] Negotiation successful
0926 04:07:27.168184 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":49,"threads_started":1}
I20260926 04:07:27.169952 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23037
I20260926 04:07:27.175558 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23112
I20260926 04:07:27.180425 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23187
2026-09-26T04:07:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2801 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 04:07:27 dist-test-slave-j8c1 krb5kdc[23404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 04:07:27 dist-test-slave-j8c1 krb5kdc[23404](info): set up 2 sockets
Sep 26 04:07:27 dist-test-slave-j8c1 krb5kdc[23404](info): commencing operation
krb5kdc: starting...
W20260926 04:07:28.226863 802 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/test-user.keytab.
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T04:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T04:07:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.200.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
Entry for principal kudu/127.0.200.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab.
I20260926 04:07:28.358319 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43175
--webserver_interface=127.0.200.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 04:07:28.444990 23417 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:28.445223 23417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:28.445241 23417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:28.445263 23417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:28.446477 23417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:28.446527 23417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:28.446561 23417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:28.446575 23417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:28.446588 23417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:28.448158 23417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.190.keytab
--principal=kudu/127.0.200.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.190:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.200.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:28.448433 23417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:28.448686 23417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:28.453605 23422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:28.453621 23425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:28.453853 23417 server_base.cc:1034] running on GCE node
W20260926 04:07:28.453634 23423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:28.454180 23417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:28.454463 23417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:28.455611 23417 hybrid_clock.cc:648] HybridClock initialized: now 1790395648455612 us; error 43 us; skew 500 ppm
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:28.458377 23417 init.cc:377] Logged in from keytab as kudu/127.0.200.190@KRBTEST.COM (short username kudu)
I20260926 04:07:28.458921 23417 webserver.cc:460] Webserver started at http://127.0.200.190:39165/ using document root <none> and password file <none>
I20260926 04:07:28.459148 23417 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:28.459211 23417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:28.459359 23417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:28.460446 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data/instance:
uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "398c38528788b899f4d40f74917c8734"
server_key_iv: "177f60b67b89bf89c3a83b47f6fd93fb"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.460814 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance:
uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "398c38528788b899f4d40f74917c8734"
server_key_iv: "177f60b67b89bf89c3a83b47f6fd93fb"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.462251 23417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260926 04:07:28.463089 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:28.463291 23417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:07:28.463408 23417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal
uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "398c38528788b899f4d40f74917c8734"
server_key_iv: "177f60b67b89bf89c3a83b47f6fd93fb"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.463495 23417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:28.480198 23417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:28.482403 23417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:28.482623 23417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:28.486707 23417 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.190:43175
I20260926 04:07:28.486763 23484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.190:43175 every 8 connection(s)
I20260926 04:07:28.487124 23417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/data/info.pb
I20260926 04:07:28.488094 23485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:28.489617 23485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.494104 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23417
I20260926 04:07:28.494222 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-0/wal/instance
I20260926 04:07:28.494422 802 external_mini_cluster.cc:1428] Setting key 13a61278ada292b3defe255ebb56ad1e
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:28.499419 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.490320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.0.200.190:43175)
0926 04:07:28.490472 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.490485 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.490587 (+ 102us) server_negotiation.cc:407] Connection header received
0926 04:07:28.491133 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.491135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.491151 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.491233 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.491767 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.492559 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.493459 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.493758 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.493842 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.496093 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.496100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.496100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.496106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.497768 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.498220 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.498222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.498222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.498296 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.498582 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.498583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.498583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.498788 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.498851 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.499110 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.499252 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260926 04:07:28.499531 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.490277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43175 (local address 127.0.0.1:44086)
0926 04:07:28.490440 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.490478 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.491008 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.491262 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.491264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.491504 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.491657 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.491661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.492650 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.492651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.493384 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.493387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.493892 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.493893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.493952 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.494576 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.494586 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.495990 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.497849 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.497857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.497860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.498138 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.498362 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.498363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.498363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.498482 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.498856 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.498863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.498951 (+ 88us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.499086 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.499346 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":68,"thread_start_us":88,"threads_started":1}
I20260926 04:07:28.500244 23485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.500682 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.500538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32798)
0926 04:07:28.500596 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.500653 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 04:07:28.500876 23435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
I20260926 04:07:28.501345 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.501229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32814)
0926 04:07:28.501281 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.501314 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 04:07:28.501467 23485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:41077: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.200.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
Entry for principal kudu/127.0.200.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab.
I20260926 04:07:28.522056 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:41077
--webserver_interface=127.0.200.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:07:28.550693 23485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } attempt: 1
I20260926 04:07:28.551258 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.551106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32816)
0926 04:07:28.551178 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.551225 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 04:07:28.551642 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.551561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32830)
0926 04:07:28.551598 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.551624 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 04:07:28.551743 23485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:41077: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
W20260926 04:07:28.607612 23492 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:28.607841 23492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:28.607861 23492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:28.607885 23492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:28.609125 23492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:28.609166 23492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:28.609185 23492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:28.609200 23492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:28.609215 23492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:28.610927 23492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.189.keytab
--principal=kudu/127.0.200.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.189:41077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.200.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:28.611145 23492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:28.611346 23492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:28.616456 23497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:28.616568 23500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:28.616622 23492 server_base.cc:1034] running on GCE node
W20260926 04:07:28.616578 23498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:28.617041 23492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:28.617347 23492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:28.618505 23492 hybrid_clock.cc:648] HybridClock initialized: now 1790395648618488 us; error 28 us; skew 500 ppm
I20260926 04:07:28.619889 23485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } attempt: 2
I20260926 04:07:28.620288 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.620155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32838)
0926 04:07:28.620210 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.620243 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 04:07:28.620705 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.620591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32844)
0926 04:07:28.620643 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.620675 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260926 04:07:28.620832 23485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.189:41077: Network error: Client connection negotiation failed: client connection to 127.0.200.189:41077: connect: Connection refused (error 111)
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:28.621357 23492 init.cc:377] Logged in from keytab as kudu/127.0.200.189@KRBTEST.COM (short username kudu)
I20260926 04:07:28.621887 23492 webserver.cc:460] Webserver started at http://127.0.200.189:43039/ using document root <none> and password file <none>
I20260926 04:07:28.622138 23492 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:28.622223 23492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:28.622378 23492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:28.623577 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data/instance:
uuid: "f8eaeb246322400fbcc24b67a452fe9a"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "02dc2f4e09efd51b9cdc02b37eab6a99"
server_key_iv: "a3db42327647696ae096a69ffdbbf4a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.623960 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal/instance:
uuid: "f8eaeb246322400fbcc24b67a452fe9a"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "02dc2f4e09efd51b9cdc02b37eab6a99"
server_key_iv: "a3db42327647696ae096a69ffdbbf4a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.625454 23492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 04:07:28.626299 23507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:28.626521 23492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 04:07:28.626649 23492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal
uuid: "f8eaeb246322400fbcc24b67a452fe9a"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "02dc2f4e09efd51b9cdc02b37eab6a99"
server_key_iv: "a3db42327647696ae096a69ffdbbf4a4"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.626776 23492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:28.644765 23492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:28.647110 23492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:28.647300 23492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:28.651484 23492 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.189:41077
I20260926 04:07:28.651515 23559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.189:41077 every 8 connection(s)
I20260926 04:07:28.651881 23492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/data/info.pb
I20260926 04:07:28.652935 23560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:28.654456 23560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.657408 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23492
I20260926 04:07:28.657487 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-1/wal/instance
I20260926 04:07:28.657680 802 external_mini_cluster.cc:1428] Setting key 28f6056423c5ff31b6f62899548140b3
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:28.663770 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.655173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.0.200.190:43175)
0926 04:07:28.655229 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.655231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.655413 (+ 182us) server_negotiation.cc:407] Connection header received
0926 04:07:28.655878 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.655879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.655890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.655957 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.656525 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.657015 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.658065 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.658279 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.658368 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.660739 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.660741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.660741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.660759 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.662129 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.662730 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.662731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.662732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.662809 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.663103 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.663104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.663104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.663207 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.663258 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.663613 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.663682 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 04:07:28.664043 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.655141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43175 (local address 127.0.0.1:44094)
0926 04:07:28.655289 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.655331 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.655775 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.656009 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.656016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.656259 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.656451 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.656455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.657090 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.657091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.657971 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.657973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.658390 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.658391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.658512 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.659119 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.659130 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.660637 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.662194 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.662201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.662203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.662608 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.662879 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.662880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.662880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.663034 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.663273 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.663279 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.663467 (+ 188us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.663599 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.663771 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260926 04:07:28.664626 23560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.200.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.200.188@KRBTEST.COM" created.
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:28.673081 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.664923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32852)
0926 04:07:28.664971 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.664983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.665045 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.665395 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.665396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.665506 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.665556 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.665557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.666890 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.666890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.667567 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.667570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.668013 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.668014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.668072 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.668444 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.668448 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.670173 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.672051 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.672052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.672052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.672207 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.672438 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.672439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.672440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.672502 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.672827 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.672828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.672865 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.672922 (+ 57us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.673006 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260926 04:07:28.673256 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.665059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.0.200.189:41077)
0926 04:07:28.665222 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.665228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.665251 (+ 23us) server_negotiation.cc:407] Connection header received
0926 04:07:28.665277 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.665279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.665295 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.665387 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.665631 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.666773 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.667646 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.667944 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.667997 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.670252 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.670261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.670262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.670269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.671958 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.672283 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.672286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.672287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.672358 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.672562 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.672563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.672563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.672757 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.672841 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.672957 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.673098 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260926 04:07:28.673884 23560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.674243 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.674131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39744)
0926 04:07:28.674176 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.674223 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 04:07:28.674414 23508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
I20260926 04:07:28.674834 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.674769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39756)
0926 04:07:28.674796 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.674819 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 04:07:28.674952 23560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38773: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
Entry for principal kudu/127.0.200.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab.
I20260926 04:07:28.685881 802 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIcgGpu/build/release/bin/kudu
/tmp/dist-test-taskIcgGpu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:38773
--webserver_interface=127.0.200.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.200.148:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 04:07:28.711979 23485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } attempt: 3
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:28.718858 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.712363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32866)
0926 04:07:28.712423 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.712436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.712501 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.712658 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.712659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.712837 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.712903 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.712905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.713572 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.713573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.714078 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.714080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.714461 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.714462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.714590 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.715048 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.715053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.716243 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.717835 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.717836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.717837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.718010 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.718203 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.718204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.718205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.718268 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.718508 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.718509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.718577 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.718643 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.718752 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260926 04:07:28.718858 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.712451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.0.200.189:41077)
0926 04:07:28.712524 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.712526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.712549 (+ 23us) server_negotiation.cc:407] Connection header received
0926 04:07:28.712581 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.712582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.712593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.712652 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.712988 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.713481 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.714161 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.714387 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.714465 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.716331 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.716333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.716333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.716341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.717738 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.718080 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.718081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.718081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.718139 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.718328 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.718329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.718329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.718445 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.718517 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.718673 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.718750 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 04:07:28.719485 23485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.719825 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.719742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39764)
0926 04:07:28.719780 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.719808 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260926 04:07:28.720135 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.720053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39768)
0926 04:07:28.720092 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.720115 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 04:07:28.720220 23485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38773: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
I20260926 04:07:28.724181 23560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } attempt: 1
I20260926 04:07:28.724665 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.724509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39774)
0926 04:07:28.724585 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.724629 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 04:07:28.725067 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.724975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39780)
0926 04:07:28.725006 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.725047 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260926 04:07:28.725989 23560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38773: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
I20260926 04:07:28.751371 23485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } attempt: 1
I20260926 04:07:28.751850 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.751708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39786)
0926 04:07:28.751776 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.751820 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 04:07:28.752166 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.752077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39792)
0926 04:07:28.752118 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.752142 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260926 04:07:28.752282 23485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38773: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
W20260926 04:07:28.769490 23567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 04:07:28.769757 23567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 04:07:28.769784 23567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 04:07:28.769817 23567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 04:07:28.771016 23567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 04:07:28.771085 23567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 04:07:28.771109 23567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 04:07:28.771138 23567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 04:07:28.771168 23567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 04:07:28.772698 23567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.200.148:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.200.190:43175,127.0.200.189:41077,127.0.200.188:38773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/krb5kdc/kudu_127.0.200.188.keytab
--principal=kudu/127.0.200.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.200.188:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.200.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 26 Sep 2026 02:52:23 UTC on ccycloud.quasar-vncnvg.root.comops.site
build id 4673638
I20260926 04:07:28.772974 23567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 04:07:28.773207 23567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 04:07:28.778064 23572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:28.778083 23575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 04:07:28.778074 23573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 04:07:28.778445 23567 server_base.cc:1034] running on GCE node
I20260926 04:07:28.778761 23567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 04:07:28.779047 23567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 04:07:28.780220 23567 hybrid_clock.cc:648] HybridClock initialized: now 1790395648780209 us; error 29 us; skew 500 ppm
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 04:07:28.782823 23567 init.cc:377] Logged in from keytab as kudu/127.0.200.188@KRBTEST.COM (short username kudu)
I20260926 04:07:28.783229 23567 webserver.cc:460] Webserver started at http://127.0.200.188:36375/ using document root <none> and password file <none>
I20260926 04:07:28.783427 23567 fs_manager.cc:359] Metadata directory not provided
I20260926 04:07:28.783468 23567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 04:07:28.783560 23567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 04:07:28.784616 23567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data/instance:
uuid: "0c7a8ba215554c70a15face48f1206ec"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "b6ff73e6e6d7f0769f020de3b89a451d"
server_key_iv: "40c2211451bb6dc293e0b7d209d0ca85"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.784935 23567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal/instance:
uuid: "0c7a8ba215554c70a15face48f1206ec"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "b6ff73e6e6d7f0769f020de3b89a451d"
server_key_iv: "40c2211451bb6dc293e0b7d209d0ca85"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.786212 23567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260926 04:07:28.786994 23582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 04:07:28.787200 23567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 04:07:28.787335 23567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data,/tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal
uuid: "0c7a8ba215554c70a15face48f1206ec"
format_stamp: "Formatted at 2026-09-26 04:07:28 on dist-test-slave-j8c1"
server_key: "b6ff73e6e6d7f0769f020de3b89a451d"
server_key_iv: "40c2211451bb6dc293e0b7d209d0ca85"
server_key_version: "encryptionkey@0"
I20260926 04:07:28.787413 23567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 04:07:28.794166 23560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } attempt: 2
I20260926 04:07:28.794632 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.794479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39804)
0926 04:07:28.794534 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.794594 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 04:07:28.794950 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.794862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39810)
0926 04:07:28.794902 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.794926 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 04:07:28.795035 23560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.200.188:38773: Network error: Client connection negotiation failed: client connection to 127.0.200.188:38773: connect: Connection refused (error 111)
I20260926 04:07:28.796501 23567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 04:07:28.798800 23567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 04:07:28.798950 23567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 04:07:28.802951 23567 rpc_server.cc:307] RPC server started. Bound to: 127.0.200.188:38773
I20260926 04:07:28.802983 23634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.200.188:38773 every 8 connection(s)
I20260926 04:07:28.803346 23567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/data/info.pb
I20260926 04:07:28.804394 23635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 04:07:28.806035 23635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.810690 802 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIcgGpu/build/release/bin/kudu as pid 23567
I20260926 04:07:28.810817 802 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIcgGpu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790395371830964-802-0/minicluster-data/master-2/wal/instance
I20260926 04:07:28.811028 802 external_mini_cluster.cc:1428] Setting key 9cd559ccccfdda5cb52827c992b06f37
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:28.816258 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.806868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.0.200.190:43175)
0926 04:07:28.806937 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.806939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.807204 (+ 265us) server_negotiation.cc:407] Connection header received
0926 04:07:28.807773 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.807775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.807786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.807847 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.808491 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.808960 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.809938 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.810133 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.810192 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.813092 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.813093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.813094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.813101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.814630 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.815116 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.815118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.815118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.815220 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.815498 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.815499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.815499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.815657 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.815738 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.816045 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.816141 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 04:07:28.816397 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.806868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.190:43175 (local address 127.0.0.1:44110)
0926 04:07:28.807072 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.807109 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.807642 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.807931 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.807935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.808196 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.808407 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.808411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.809039 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.809040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.809851 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.809853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.810221 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.810222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.810347 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.810966 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.810974 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.812978 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.814726 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.814733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.814735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.815043 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.815310 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.815311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.815311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.815425 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.815771 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.815777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.815905 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.816004 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.816130 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:28.816977 23635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } has no permanent_uuid. Determining permanent_uuid...
I20260926 04:07:28.820468 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.811484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.0.200.188:38773)
0926 04:07:28.811674 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.811680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.811707 (+ 27us) server_negotiation.cc:407] Connection header received
0926 04:07:28.811731 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.811734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.811789 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.811865 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.812631 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.813491 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.814243 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.814594 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.814681 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.817415 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.817424 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.817424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.817432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.819122 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.819523 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.819526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.819526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.819590 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.819803 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.819803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.819804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.819989 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.820156 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.820253 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.820340 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:28.824225 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.817370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.0.200.189:41077)
0926 04:07:28.817424 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.817426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.817468 (+ 42us) server_negotiation.cc:407] Connection header received
0926 04:07:28.817535 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.817536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.817548 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.817621 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.817865 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.818325 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.819022 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.819310 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.819385 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.821352 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.821353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.821354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.821361 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.823036 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.823435 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.823436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.823436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.823491 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.823733 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.823734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.823734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.823861 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.823914 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.824061 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.824123 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:07:28.824257 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.817295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.189:41077 (local address 127.0.0.1:32878)
0926 04:07:28.817422 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.817432 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.817477 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.817620 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.817621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.817734 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.817787 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.817788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.818397 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.818399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.818942 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.818944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.819388 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.819389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.819451 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.819887 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.819892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.821279 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.823127 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.823128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.823129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.823353 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.823553 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.823553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.823554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.823667 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.823922 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.823923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.823969 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.824043 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.824141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260926 04:07:28.824663 23635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.190@KRBTEST.COM
I20260926 04:07:28.827432 23485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } attempt: 2
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.188@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:28.830420 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.821294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.0.200.190:43175)
0926 04:07:28.821579 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.821581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.821587 (+ 6us) server_negotiation.cc:407] Connection header received
0926 04:07:28.821608 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.821609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.821617 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.821659 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.822420 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.823232 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.824215 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.824493 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.824573 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.827400 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.827402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.827402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.827413 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.829240 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.829620 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.829621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.829622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.829679 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.829884 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.829884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.829885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.830032 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.830209 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.830245 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.830317 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260926 04:07:28.832624 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.824957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39824)
0926 04:07:28.825120 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.825128 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.825156 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.825267 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.825267 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.825379 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.825413 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.825414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.826347 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.826348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.827186 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.827189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.827622 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.827623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.827684 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.828109 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.828114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.829432 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.831501 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.831502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.831503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.831723 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.831931 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.831932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.831932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.831986 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.832265 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.832266 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.832298 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.832370 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.832471 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260926 04:07:28.832931 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.825079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.0.200.188:38773)
0926 04:07:28.825136 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.825138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.825163 (+ 25us) server_negotiation.cc:407] Connection header received
0926 04:07:28.825193 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.825194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.825205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.825265 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.825476 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.826248 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.827265 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.827498 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.827570 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.829512 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.829513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.829513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.829520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.831429 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.831809 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.831810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.831811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.831861 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.832064 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.832064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.832064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.832187 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.832228 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.832412 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.832529 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":36}
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.190@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:28.834772 23635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: Bootstrap starting.
I20260926 04:07:28.835471 23635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:28.835767 23635 log.cc:824] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: Log is configured to *not* fsync() on all Append() calls
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.200.189@KRBTEST.COM
I20260926 04:07:28.836860 23635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: No bootstrap required, opened a new log
I20260926 04:07:28.837066 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.827704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39828)
0926 04:07:28.827893 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.827904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.827944 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.828435 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.828436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.828658 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.828707 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.828710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.829724 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.829725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.830599 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.830602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.832134 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.832135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.832197 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.832572 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.832577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.833991 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.835960 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.835961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.835962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.836188 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.836394 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.836395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.836395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.836459 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.836767 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.836768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.836814 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.836863 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.836968 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260926 04:07:28.837548 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.827764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.0.200.188:38773)
0926 04:07:28.828135 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.828137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.828143 (+ 6us) server_negotiation.cc:407] Connection header received
0926 04:07:28.828158 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.828159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.828167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.828255 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.828793 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.829659 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.830679 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.832057 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.832135 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.834104 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.834105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.834105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.834111 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.835878 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.836262 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.836263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.836263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.836314 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.836527 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.836534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.836535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.836668 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.837254 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.837294 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.837373 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":92,"threads_started":1}
I20260926 04:07:28.839135 23635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.839300 23635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7a8ba215554c70a15face48f1206ec, State: Initialized, Role: FOLLOWER
I20260926 04:07:28.839404 23485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: Bootstrap starting.
I20260926 04:07:28.839386 23635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.839684 23635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:28.839648 23642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.839874 23642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.840076 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.831340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.0.200.189:41077)
0926 04:07:28.831391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.831393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.831428 (+ 35us) server_negotiation.cc:407] Connection header received
0926 04:07:28.831507 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.831508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.831517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.831567 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.832201 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.833042 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.834028 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.834313 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.834387 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.837178 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.837179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.837180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.837187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.838791 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.839139 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.839140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.839141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.839201 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.839402 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.839403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.839404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.839530 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.839676 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.839900 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.839983 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260926 04:07:28.840118 23485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:28.840454 23485 log.cc:824] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:28.841305 23485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: No bootstrap required, opened a new log
I20260926 04:07:28.841650 802 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 04:07:28.842458 23459 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44120
W20260926 04:07:28.843145 23654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:28.843190 23654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:28.843405 23485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.843492 23485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7530c1f3e1214438b0fc5b4d18f3dc3e, State: Initialized, Role: FOLLOWER
I20260926 04:07:28.843539 23485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.843752 23656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.843812 23485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:28.843847 23459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 04:07:28.843906 23459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:28.842075 23567 master_runner.cc:186] Error getting master registration for 127.0.200.190:43175: OK, instance_id { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" instance_seqno: 1790395648485512 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:07:28.844805 23459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.844869 23567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:07:28.845064 23459 leader_election.cc:288] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [CANDIDATE]: Term 1 election: Requested vote from peers f8eaeb246322400fbcc24b67a452fe9a (127.0.200.189:41077), 0c7a8ba215554c70a15face48f1206ec (127.0.200.188:38773)
I20260926 04:07:28.845500 23609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0c7a8ba215554c70a15face48f1206ec"
I20260926 04:07:28.845613 23609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [term 0 FOLLOWER]: Advancing to term 1
W20260926 04:07:28.845791 23667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:28.846369 23667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:28.846503 23609 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7530c1f3e1214438b0fc5b4d18f3dc3e in term 1.
I20260926 04:07:28.846740 23433 leader_election.cc:302] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7a8ba215554c70a15face48f1206ec, 7530c1f3e1214438b0fc5b4d18f3dc3e; no voters:
I20260926 04:07:28.843816 23656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.846982 23656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 04:07:28.847055 23656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [term 1 LEADER]: Becoming Leader. State: Replica: 7530c1f3e1214438b0fc5b4d18f3dc3e, State: Running, Role: LEADER
I20260926 04:07:28.847141 23656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.847563 23656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7530c1f3e1214438b0fc5b4d18f3dc3e. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.847641 23656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:28.847154 23417 master_runner.cc:186] Error getting master registration for 127.0.200.189:41077: OK, instance_id { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" instance_seqno: 1790395648650241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 04:07:28.849138 23417 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 04:07:28.851482 23676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 04:07:28.851749 23676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 04:07:28.858244 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.850823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.0.200.190:43175)
0926 04:07:28.850870 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.850872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.851326 (+ 454us) server_negotiation.cc:407] Connection header received
0926 04:07:28.851398 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.851399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.851410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.851460 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.852165 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.852731 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.853392 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.853673 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.853738 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.855288 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.855289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.855289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.855297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.856978 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.857367 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.857368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.857369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.857427 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.857641 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.857642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.857642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.857806 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.857928 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.858069 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.858148 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260926 04:07:28.861286 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.846251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.0.200.189:41077)
0926 04:07:28.846291 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.846293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.851066 (+ 4773us) server_negotiation.cc:407] Connection header received
0926 04:07:28.851226 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.851227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.851250 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.851282 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.852470 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.853195 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.854778 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.855087 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.855131 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.857223 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.857224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.857224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.857231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.859244 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.859806 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.859807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.859808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.859864 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.860133 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.860134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.860134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.860277 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.860786 (+ 509us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.861102 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.861186 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260926 04:07:28.866210 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.851108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.0.200.188:38773)
0926 04:07:28.851151 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.851153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.858691 (+ 7538us) server_negotiation.cc:407] Connection header received
0926 04:07:28.858809 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.858810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.858825 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.858880 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.859516 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.860263 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.861251 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.861576 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.861618 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.863191 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.863192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.863193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.863200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.864835 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.865208 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.865209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.865209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.865284 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.865500 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.865501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.865502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.865662 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.865783 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.865896 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.866029 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260926 04:07:28.866312 23609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [term 1 FOLLOWER]: Refusing update from remote peer 7530c1f3e1214438b0fc5b4d18f3dc3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:28.866724 23674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 04:07:28.868021 23642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7530c1f3e1214438b0fc5b4d18f3dc3e. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.868124 23642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.871932 23674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.872032 23674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:28.872172 23642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.872254 23642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.872325 23680 mvcc.cc:205] Tried to move back new op lower bound from 7333460577754308608 to 7333460577679450112. Current Snapshot: MvccSnapshot[applied={T|T < 7333460577754308608}]
I20260926 04:07:28.878986 23676 catalog_manager.cc:1347] Generated new cluster ID: ced8ee2396154a5d9d7b71cd501fe468
I20260926 04:07:28.879047 23676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 04:07:28.883836 23676 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 04:07:28.884557 23676 catalog_manager.cc:1504] Loading token signing keys...
I20260926 04:07:28.886149 23560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } attempt: 3
Sep 26 04:07:28 dist-test-slave-j8c1 krb5kdc[23404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790395648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.200.189@KRBTEST.COM for kudu/127.0.200.188@KRBTEST.COM
I20260926 04:07:28.894635 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.886420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.200.188:38773 (local address 127.0.0.1:39850)
0926 04:07:28.886463 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0926 04:07:28.886474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 04:07:28.886598 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 04:07:28.886735 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 04:07:28.886736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 04:07:28.886922 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0926 04:07:28.886972 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.886974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.887804 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.887805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.888629 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 04:07:28.888632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.889139 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.889140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 04:07:28.889208 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.889722 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 04:07:28.889727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 04:07:28.891305 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 04:07:28.893474 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.893475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.893475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.893714 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.893930 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 04:07:28.893930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 04:07:28.893931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 04:07:28.893989 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 04:07:28.894304 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 04:07:28.894305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 04:07:28.894357 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 04:07:28.894422 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 04:07:28.894519 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 04:07:28.894666 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:28.886450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.0.200.188:38773)
0926 04:07:28.886498 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.886500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.886509 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:28.886663 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.886664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.886673 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.886732 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.887031 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.887727 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.888734 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:28.889037 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:28.889126 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:28.891401 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 04:07:28.891402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 04:07:28.891403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 04:07:28.891410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0926 04:07:28.893370 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.893777 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.893778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.893778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.893862 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 04:07:28.894074 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 04:07:28.894076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 04:07:28.894076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0926 04:07:28.894239 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 04:07:28.894308 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:28.894471 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 04:07:28.894580 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 04:07:28.897361 23560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: Bootstrap starting.
I20260926 04:07:28.898102 23560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: Neither blocks nor log segments found. Creating new log.
I20260926 04:07:28.898401 23560 log.cc:824] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: Log is configured to *not* fsync() on all Append() calls
I20260926 04:07:28.899214 23560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: No bootstrap required, opened a new log
I20260926 04:07:28.901168 23560 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.901314 23560 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8eaeb246322400fbcc24b67a452fe9a, State: Initialized, Role: FOLLOWER
I20260926 04:07:28.901417 23560 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } }
I20260926 04:07:28.901754 23560 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 04:07:28.901829 23534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f8eaeb246322400fbcc24b67a452fe9a"
I20260926 04:07:28.901952 23534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [term 0 FOLLOWER]: Advancing to term 1
I20260926 04:07:28.902845 23534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7530c1f3e1214438b0fc5b4d18f3dc3e in term 1.
I20260926 04:07:28.903080 23533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [term 1 FOLLOWER]: Refusing update from remote peer 7530c1f3e1214438b0fc5b4d18f3dc3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 04:07:28.903694 23685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.903801 23685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.903923 23674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 04:07:28.904541 23696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 04:07:28.904595 23696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 04:07:28.908313 23700 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7530c1f3e1214438b0fc5b4d18f3dc3e. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.908414 23700 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.908450 23700 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.908515 23700 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 04:07:28.908474 23656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7530c1f3e1214438b0fc5b4d18f3dc3e" member_type: VOTER last_known_addr { host: "127.0.200.190" port: 43175 } } peers { permanent_uuid: "f8eaeb246322400fbcc24b67a452fe9a" member_type: VOTER last_known_addr { host: "127.0.200.189" port: 41077 } } peers { permanent_uuid: "0c7a8ba215554c70a15face48f1206ec" member_type: VOTER last_known_addr { host: "127.0.200.188" port: 38773 } } }
I20260926 04:07:28.908557 23656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e [sys.catalog]: This master's current role is: LEADER
I20260926 04:07:28.910629 23676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7530c1f3e1214438b0fc5b4d18f3dc3e: Generated new TSK 0
I20260926 04:07:28.910867 23676 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 04:07:28.942993 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.941482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.0.200.189:41077)
0926 04:07:28.941521 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.941523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.942230 (+ 707us) server_negotiation.cc:407] Connection header received
0926 04:07:28.942343 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.942344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.942356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.942387 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.942905 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32902: BlockingRecv error: recv got EOF from 127.0.0.1:32902 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 04:07:28.943737 23640 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.941229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.0.200.188:38773)
0926 04:07:28.941278 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.941280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.943248 (+ 1968us) server_negotiation.cc:407] Connection header received
0926 04:07:28.943353 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.943354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.943364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.943393 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.943680 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39860: BlockingRecv error: recv got EOF from 127.0.0.1:39860 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260926 04:07:28.944607 23488 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.941683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.0.200.190:43175)
0926 04:07:28.941719 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.941721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.943958 (+ 2237us) server_negotiation.cc:407] Connection header received
0926 04:07:28.944107 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.944108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.944119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.944184 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.944544 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44140: BlockingRecv error: recv got EOF from 127.0.0.1:44140 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260926 04:07:28.946907 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.946229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.0.200.189:41077)
0926 04:07:28.946275 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.946277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.946390 (+ 113us) server_negotiation.cc:407] Connection header received
0926 04:07:28.946494 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.946495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.946513 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.946562 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.946859 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32910: BlockingRecv error: recv got EOF from 127.0.0.1:32910 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260926 04:07:28.949982 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.949222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.0.200.189:41077)
0926 04:07:28.949256 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.949258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.949514 (+ 256us) server_negotiation.cc:407] Connection header received
0926 04:07:28.949600 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.949601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.949612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.949639 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.949937 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32926: BlockingRecv error: recv got EOF from 127.0.0.1:32926 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 04:07:28.953917 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.953129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.0.200.189:41077)
0926 04:07:28.953165 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.953167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.953423 (+ 256us) server_negotiation.cc:407] Connection header received
0926 04:07:28.953520 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.953521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.953531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.953560 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.953871 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32942: BlockingRecv error: recv got EOF from 127.0.0.1:32942 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260926 04:07:28.960080 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.959157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.0.200.189:41077)
0926 04:07:28.959210 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.959212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.959466 (+ 254us) server_negotiation.cc:407] Connection header received
0926 04:07:28.959628 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.959629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.959641 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.959672 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.960030 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32952: BlockingRecv error: recv got EOF from 127.0.0.1:32952 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260926 04:07:28.970285 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.969338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.0.200.189:41077)
0926 04:07:28.969382 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.969384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.969668 (+ 284us) server_negotiation.cc:407] Connection header received
0926 04:07:28.969780 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.969781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.969794 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.969826 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.970226 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32966: BlockingRecv error: recv got EOF from 127.0.0.1:32966 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 04:07:28.988648 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:28.987670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.0.200.189:41077)
0926 04:07:28.987720 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:28.987723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:28.988019 (+ 296us) server_negotiation.cc:407] Connection header received
0926 04:07:28.988128 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:28.988129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:28.988143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:28.988189 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:28.988584 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32974: BlockingRecv error: recv got EOF from 127.0.0.1:32974 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260926 04:07:29.022935 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:29.021966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.0.200.189:41077)
0926 04:07:29.022011 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.022013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.022292 (+ 279us) server_negotiation.cc:407] Connection header received
0926 04:07:29.022391 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.022392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.022404 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.022449 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:29.022869 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32980: BlockingRecv error: recv got EOF from 127.0.0.1:32980 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260926 04:07:29.088950 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:29.088237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.0.200.189:41077)
0926 04:07:29.088331 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.088333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.088381 (+ 48us) server_negotiation.cc:407] Connection header received
0926 04:07:29.088424 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.088424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.088434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.088510 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:29.088879 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32992: BlockingRecv error: recv got EOF from 127.0.0.1:32992 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 04:07:29.218883 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:29.218085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.0.200.189:41077)
0926 04:07:29.218161 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.218164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.218240 (+ 76us) server_negotiation.cc:407] Connection header received
0926 04:07:29.218336 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.218337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.218352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.218422 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:29.218801 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33006: BlockingRecv error: recv got EOF from 127.0.0.1:33006 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 04:07:29.476902 23564 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 04:07:29.476270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.0.200.189:41077)
0926 04:07:29.476378 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.476380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.476389 (+ 9us) server_negotiation.cc:407] Connection header received
0926 04:07:29.476420 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.476421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.476433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.476505 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 04:07:29.476852 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33012: BlockingRecv error: recv got EOF from 127.0.0.1:33012 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 04:07:29.843638 23654 catalog_manager.cc:1259] Loaded cluster ID: ced8ee2396154a5d9d7b71cd501fe468
I20260926 04:07:29.843720 23654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: loading cluster ID for follower catalog manager: success
I20260926 04:07:29.844662 23654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: acquiring CA information for follower catalog manager: success
I20260926 04:07:29.844928 23654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c7a8ba215554c70a15face48f1206ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:07:29.905049 23696 catalog_manager.cc:1259] Loaded cluster ID: ced8ee2396154a5d9d7b71cd501fe468
I20260926 04:07:29.905164 23696 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: loading cluster ID for follower catalog manager: success
I20260926 04:07:29.906008 23696 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: acquiring CA information for follower catalog manager: success
I20260926 04:07:29.906324 23696 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8eaeb246322400fbcc24b67a452fe9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 04:07:29.993606 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:29.990311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.0.200.189:41077)
0926 04:07:29.990473 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.990475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.990489 (+ 14us) server_negotiation.cc:407] Connection header received
0926 04:07:29.990595 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.990596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.990677 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.990753 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:29.990959 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:29.991487 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:29.992667 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:29.992863 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:29.992922 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:29.993140 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:29.993333 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:29.993406 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:29.993482 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 04:07:29.993502 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260926 04:07:29.998715 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:29.995515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.0.200.188:38773)
0926 04:07:29.995654 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:29.995656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:29.995669 (+ 13us) server_negotiation.cc:407] Connection header received
0926 04:07:29.995748 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:29.995749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:29.995810 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:29.995902 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:29.996084 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:29.996788 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:29.997647 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:29.998019 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:29.998079 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:29.998275 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:29.998392 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:29.998469 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:29.998607 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 04:07:29.998635 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260926 04:07:30.003420 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:30.000390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.0.200.189:41077)
0926 04:07:30.000444 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:30.000446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:30.000662 (+ 216us) server_negotiation.cc:407] Connection header received
0926 04:07:30.000763 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:30.000764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:30.000776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:30.000807 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:30.000975 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.001552 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.002521 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.002844 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.002909 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:30.003110 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:30.003144 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:30.003184 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:30.003313 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 04:07:30.003338 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260926 04:07:30.003883 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:30.000421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.0.200.190:43175)
0926 04:07:30.000534 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:30.000536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:30.000681 (+ 145us) server_negotiation.cc:407] Connection header received
0926 04:07:30.000796 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:30.000797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:30.000851 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:30.000907 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:30.001198 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.001907 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.002881 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.003094 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.003166 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:30.003361 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:30.003477 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:30.003553 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:30.003772 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 04:07:30.003800 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260926 04:07:30.004848 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 04:07:30.000584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.0.200.188:38773)
0926 04:07:30.000620 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0926 04:07:30.000622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0926 04:07:30.000633 (+ 11us) server_negotiation.cc:407] Connection header received
0926 04:07:30.000661 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 04:07:30.000661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 04:07:30.000672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 04:07:30.000727 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 04:07:30.000828 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.001550 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.003482 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 04:07:30.004221 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 04:07:30.004313 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 04:07:30.004593 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 04:07:30.004621 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 04:07:30.004664 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 04:07:30.004747 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 04:07:30.004767 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260926 04:07:30.013211 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23417
I20260926 04:07:30.019183 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23492
I20260926 04:07:30.024215 802 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIcgGpu/build/release/bin/kudu with pid 23567
2026-09-26T04:07:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2842 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5643 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278199 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST